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PREFACE 


IDA  Paper  P-2423,  Improving  Diagnostics  Capabilities  Through  Enhanced  Maintenance 
Data  Collection,  was  prepared  for  the  Weapons  Support  Improvement  Group  (WSIG),  Office  of 
the  Assistant  Secretary  of  Defense  for  Production  and  Logistics  (OASD(P&L))  in  response  to  IDA 
Task  Order  T-B5-490  under  MDA  903-89C-0003.  The  study  reported  here  was  initiated  as  a  direct 
result  of  the  observations  and  recommendations  from  the  Joint  DoDIIndustry  Study  on 
Opportunities  in  Integrated  Diagnostics  [Brown  1990]. 

This  paper  reviews  the  scope  and  capabilities  of  the  DoD  maintenance  data  collection 
(MDC)  systems,  compares  these  capabilities  with  those  found  in  the  commercial  sector  during  the 
Joint  DoD/Industiy  Study,  and  recommends  near  and  long  term  actions  to  enhance  DoD’s 
capabilities  to  diagnose  and  troubleshoot  military  equipment. 

The  paper  was  reviewed  by  the  members  of  the  following  CSED  Peer  Review:  Richard 
Ivanetich,  Richard  Wexelblat,  Terry  Mayfield,  Robert  Winner,  James  Pennell,  David  Dierolf,  and 
Karen  Richter.  We  also  express  our  appreciation  to  Katydean  Price  and  Sylvia  Reynolds  for  their 
editorial  advice  and  assistance,  and  to  Betty  Pinna  and  Janice  Hirst  for  their  word  processing  and 
graphics  support.  Finally,  we  thank  Chris  Fisher  and  Marty  Meth  of  OASD(P&L)  WSIG  for  their 
guidance  and  support. 


V 


EXECUTIVE  SUMMARY 


Many  studies  have  reported  that  it  is  hard  to  quickly  and  accurately  diagnose  military 
equipment  failures.  Field  maintenance  accuracy  today  relies  on  highly  specialized  skills,  laige 
complex  automatic  test  equipment,  on-board  built-in  tests  (that  are  often  inadequate),  and  outdated 
technical  information.  The  results  are  that  significant  DoD  resources  are  tied  up  in  unnecessary 
maintenance  actions  and  parts  removals,  high  manpower  levels,  and  extensive  technical  training. 
Commercial  companies  reviewed  in  an  earlier  IDA  study,  the  Joint  DoD! Industry  Study  on 
Opportunities  in  Integrated  Diagnostics  [Brown  1990],  were  found  to  purposefully  collect  and 
apply  diagnostic  related  maintenance  information  to  greatly  enhance  diagnostic  performance  and 
cost  effectiveness  for  their  products. 

The  study  reported  in  this  paper  examined  primary,  large  scale  DoD  maintenance  data 
collection  (MDC)  systems  to  determine  if  they  could  or  should  be  modified  to  enhance  the  use  of 
information  in  order  to  improve  diagnostics.  We  found  that  the  MDC  systems  in  place,  or  now 
being  implemented,  have  principally  automated  previous  manual  functions.  While  we  observed 
that  the  products  of  these  MDC  systems  are  not  intended  for  diagnostic  functions,  the  net  result  is 
that  these  enhancements  have  not  improved  DoD  diagnostic  capabilities.  Furthermore,  these 
systems  rarely  provide  any  direct  benefit  to  the  maintenance  technician. 

We  believe  that  very  significant  diagnostic  improvements  are  practical  if  further  steps  are 
taken  to  integrate  the  processes  of  maintenance,  data  collection,  analysis,  and  information 
feedback. 

CURRENT  SITUATION 

Today’s  Service  MDC  systems  are  designed  to  collect  irrformation  for  maintenance 
management  activities  and  to  provide  hardware  usage  information  for  the  supply  systems.  Over  a 
ten  year  period  more  than  a  billion  dollars  will  have  been  spent  for  DoD  MDC  systems 
improvements  to  change  the  manual,  paper  based  systems  to  automated  data  processing  (ADP) 
systems.  The  principal  result  of  this  effort  is  the  interconnection  of  existing  MDC  and  logistics 
data  bases. 

These  systems  carry  over  many  of  the  problems  of  the  manual  systems  they  replace.  The 
normal  benefits  of  automation  are  not  being  fully  realized  because  of  the  continued  need  for 
manpower-intensive  data  input  methods  and  the  limitations  of  the  hardware  and  software  hosting 
the  MDC  systems.  Rarely  is  the  MDC  information  used  to  support  or  improve  diagnostics;  in  fact, 
we  found  no  organizations  that  acknowledge  responsibility  for  the  purposeful  use  of  MDC  data  to 
improve  diagnostics  as  a  primary  mission. 
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In  contrast,  maintenance  diagnostic  capabilities  of  the  commercial  companies  investigated 
in  the  Joint  DoDUndustry  Study  on  Opportunities  in  Integrated  Diagnostics  were  found  to 
specifically  and  effectively  use  MDC  data  to  enhance  diagnostics  capabilities.  The  commercial 
MDC  systems  transparently  acquire  maintenance  data,  analyze  it  pronq)tly,  and  rapidly 
disseminate  updated  diagnostics  information  to  the  maintainer.  Further,  commercial  companies 
commonly  assign  the  responsibility  for  all  aspects  of  diagnostics,  including  management  of  MDC 
systems  to  a  single  organization. 

CONCLUSIONS 

The  Service  approaches  to  providing  and  managing  diagnostic  capabilities  are 
fractionated  and  this  problem  is  particularly  evident  in  MDC  implementation.  Improvements  to 
diagnostics  capabilities  will  require  more  than  just  changes  in  mechanics  or  scope  of  MDC 
systems.  Fundamental  changes  in  maintenance  capability  development  and  process  management 
will  be  needed.  Because  of  the  substantial  investments  already  made  in  fielding  current 
“modernized”  MDC  systems,  and  because  many  of  the  MDC  information  products  are  justified 
and  targeted  for  needed  maintenance  management  and  supply  support  functions,  a  focused  study 
is  needed  before  any  major  changes  are  undertaken.  The  study  should  look  at  both  near  and  long 
term  opportunities  to  effectively  integrate  MDC  capabilities  with  diagnostics  improvement 
processes  that  will  produce  substantial  increases  in  maintenance  efficiency. 

RECOMMENDED  ACTIONS 

In  the  near  term,  the  following  actions  are  recommended  to  establish  an  environment  for 
diagnostic  improvement  and  to  improve  the  efficiency  and  effectiveness  of  existing  MDC 
capabilities; 

a.  Assign  diagnostics  and  maintenance  accountability  to  individuals  presently  working 
within  each  existing  weapon  system  maintenance  and  engineering  organization.  This 
wiU  establish  a  foundation,  at  the  individual  systems  and  subsystems  organizations, 
where  MDC  information  may  be  effectively  analyzed  and  put  to  productive  use  for 
identifying  diagnostic  improvement  needs. 

b.  Use  available  technology  to  automate  point  of  entry  data  capture  (e.g.,  F-18  engine  and 
avionics  data  arutlysis,  F-IS  pilot  debrief  aids).  These  actions  will  improve  data 
accuracy  needed  to  support  analyses  and  enhance  the  data  integrity  for  existing  MDC 
supported  functions. 


c.  Provide  automated  links  between  MDC  and  new  automatic  test  equipment  (ATE) 
systems.  These  actions  will  help  to  reduce  data  collection  work  loads  and  data  entry 


errors.  Pnirthermore,  they  will  provide  essential  links  for  future  data  analyses  across 
functional,  maintenance,  and  product  lines. 

d.  Encourage  system  contractors  to  propose  new  MDC  capabilities  or  modifications  to 
existing  systems  with  the  objective  of  increasing  diagnostics  capability.  These  actions 
v/ill  help  stimulate  creative  ideas  and  promote  a  move  away  from  the  status  quo  to  a 
more  integrated  approach. 

For  the  long  term,  we  recommend  that  the  approach  to  management  of  maintenance 
diagnostics  be  reexamined.  The  suggested  approach  for  DoD  is  outlined  in  our  previous  Joint 
DoDIIndustry  Study  on  Opportunities  in  Integrated  Diagnostics.  In  this  report  we  recommended 
that  DoD  develop  a  planning  framework  to  facilitate  the  implementation  of  integrated  diagnostic 
concepts.  This  planning  framework  should  specifically  address  actions  necessary  to  accelerate 
weapon  system  specific  point  of  entry  data  capture  for  increasing  data  accuracy  and  completeness, 
to  extend  common  MDC  ATE  systems  for  collecting  and  disseminating  applicable  diagnostic- 
related  information,  and  to  establish  requirements  and  incentives  for  acquiring  continuously 
improving  designed-in  MDC  capabilities.  We  further  suggest  that  while  this  examination  is  going 
on,  the  Services  not  start  any  further  enhancements  to  MDC  systems  without  reviewing  the 
opportunities  of  more  efficient  integrated  process  approaches  found  in  the  commercial  cases. 
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1.  INTRODUCTION 


1.1  PURPOSE 

The  purpose  of  this  study  was  to  examine  Department  of  Defense  (DoD)  maintenance  data 
collection  (MDC)  systems  to  determine  if  they  could  and  should  be  modified  to  improve 
diagnosing  and  troubleshooting  military  equipment.  The  study,  requested  by  the  Weapon  Support 
Improvement  Group  (WSIG),  Office  of  the  Assistant  Secretary  of  Defense  for  Production  and 
Lx)gistics  (OASD(P&L)),  examined  selected  DoD  MDC  systems.  Study  objectives  were  to  assess 
how  effectively  DoD  uses  the  collected  data  to  improve  diagnostic  accuracy,  to  conq)are  DoD 
practices  with  commercial  MDC  implementations  that  support  integrated  diagnostics,  and  to 
identify  opportunities  to  improve  DoD’s  weapons  systems  diagnostic  capabilities  through 
enhanced  application  of  MDC  systems. 


1.:  BACKGROUND 

A  large  fraction  (estimated  25  to  30  percent)  of  a  weapon  system’s  lifecycle  cost  is  spent 
on  maintenance.  Approximately  50  percent  of  this  maintenance  cost  comes  from  the  diagnostic 
processes  of  locating  problems  and  identifying  potential  solutions!  Maintenance  effectiveness  of 
field  and  depot  activities  is  directly  related  to  the  speed,  efficiency,  and  accuracy  of  the  diagnostic 
process.  The  complexity  of  today’s  weapon  systems  has  led  to  the  establishment  of  field 
maintenance  environments  that  require  large  logistics  infrastructures.  These  infrastructures  need 
specially  trained  personnel,  dedicated  complex  test  and  support  equipment,  and  volumes  of 
tecluucal  manuals. 

DoD  maintenance  communities  believe  that  the  existing  diagnostic  capabilities  can  and 
must  be  improved.  False  maintenance  removal  rates  of  between  twenty  and  fifty  percent  for 
electronic  equipment  are  common.  However,  these  problems  are  not  limited  to  the  electronic 
equipment.  For  example,  analyses  of  data  for  unscheduled  maintenance  of  F-16A/B  aircraft  at 
eight  Air  Force  bases  in  1985  revealed  the  following  percentages  of  “no-defect-found”:  flight 
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controls:  18%,  turbofan  power  plant:  16%,  fuel  system:  23%,  fire  control:  49%,  weapons  deliveiy; 

47%,  and  penetration  aids  and  electronic  counter  measures:  19%  [Petruschell  1987,p.30]. 

WSIG  sponsored  workshops  in  June  and  August  1989  to  identify  opportunities  in 

integrated  diagnostics.  In  this  paper,  the  term  integrated  diagnostics  is  used  to  represent  a  process 

of  improving  diagnostic  effectiveness  and  maintenance  efficiency  through  the  integration  of 

pertinent  elements  such  ps  testability,  automatic  and  manual  testing,  training,  maintenance  aiding, 

and  technical  information.  Case  studies  presented  during  these  workshops  highlighted  successful 

commercial  applications  of  integrated  diagnostics. 

The  case  studies  illustrated  dramatic  improvements  in  maintenance  productivity 
and  operating  efficiency:  (1)  The  General  Motors  Computerized  Automotive 
Maintenance  System  demonstrated  33%  improvement  in  maintenance  shop  pro¬ 
ductivity  within  3  weeks  of  usage,  (2)  the  AT&T  5ESS  electronic  switching  sys¬ 
tem  automated  maintenance  performance  exceeds  the  system  requirements  of  less 
than  3  minutes  of  down  time  per  year,  and  (3)  the  General  Electric  Ground-based 
[aircraft]  Engine  Monitoring  system  on-wing  (Lufthansa)  maintenance  perfor¬ 
mance  analyses  reduces  maintenance  overhaul  budgets  by  5%  and  reduces  fuel 
consumption  by  an  estimated  0.5%.  [Brown  1990,  p.vii.] 

The  key  attributes  found  in  these  commercial  examples,  often  contrasted  with  the 

perception  of  current  DoD  practices  and  guidelines  [Brown  1990,pp.ll-15].  The  workshop 

participants  observed  that  commercial  systems  with  highly  effective  diagnostics  capabilities 

typically  employed  effective  MDC,  centralized  data  analyses,  and  rapid  feedback  of  critical 

information  to  both  the  design  and  maintenance  communities.  They  went  on  to  observe: 

Effective  data  feedback  loops  to  the  maintainers  and  designers  are  essential  to 
successful  integrated  diagnostics  in^lementation.  However,  the  effectiveness  and 
accuracy  of  field  maintenance  data  collection  for  DoD  systems  are  generally  less 
capable  than  those  illustrated  in  the  case  studies.  DoD  field  units  currently  collect 
enormous  amounts  of  data  that  are  not  being  analyzed  effectively  [Brown  1990, 
p.89]. 

The  data  collected  by  the  DoD  MDC  systems  are  incident  to  the  maintenance  and  systems 
diagnostics  processes,  and  not  intended  to  specifically  support  systems  diagnostics  or  improve 
diagnostics  capabilities.  Although  the  information  is  collected  for  a  variety  of  reasons  that  are  not 
directly  tied  to  the  diagnostics  needs,  there  was  strong  consensus  of  the  workshop  participants  that 
the  available  MDC  information  could  be  used  to  improve  DoD  diagnostics  performance. 

The  study  reported  here  was  initiated  as  a  direct  result  of  the  workshop  observations.  It  is 
intended  to  focus  on  how  DoD  MDC  data,  as  well  as  the  related  capabilities,  may  be  used  to 
improve  diagnostics.  Additional  background  information  has  been  included  in  Appendix  A, 
“Discussion  of  Terms,”  to  provide  a  common  framework  for  the  reader  of  this  paper. 


2.  APPROACH 


2.1  SCOPE 

At  the  onset  of  the  study,  we  recognized  that  the  DoD  MDC  systems  have  been  established 
for  purposes  other  than  improving  system  diagnostics  performance  and  accuracy.  However,  as 
observed  by  the  participants  in  the  Integrated  Diagnostics  Workshops,  information  collected  by 
these  MDC  systems  often  finds  its  way  back  into  various  diagnostics  element  domains  (training, 
documentation,  reliability  and  availability  data  bases,  etc.).  As  this  information  collects,  actions 
that  can  influence  diagnostics  performance  may  be  more  easily  analyzed  and  identified. 

We  also  noted  that  the  Services  collect  huge  amounts  of  maintenance  data  by  a  variety  of 
MDC  systems.  Therefore,  in  order  to  limit  the  scope,  several  DoD  MDC  systems,  existing  and 
under  development,  were  identified  for  this  study.  The  criteria  for  selection  included  (1)  the 
systems  selected  must  be  representative  of  standard  (or  proposed  future  standard)  MDC  systems 
for  a  broad  range  of  weapon  systems,  (2)  the  information  collected  or  analyzed  by  the  MDC 
systems  must  be  representative  of  the  type  and  range  of  data  needed  by  the  Service,  and  (3)  the 
MDC  systems  should  relate  to  operational  weapon  systems  so  that  integrated  diagnostics 
opportunities  could  be  assessed. 


2.2  SYSTEMS  AND  PROGRAMS  EVALUATED 

Table  1  presents  a  summary  of  the  systems  and  programs  evaluated  during  this  study. 
Purpose  and  status  summaries  for  each  of  these  programs  are  presented  by  Service  in  Thbles  2,3, 
and  4.  Specific  details  regarding  the  individual  systems  and  programs  evaluated  are  presented  in 
Appendix  B,  "Summary  of  Systems  and  Programs  Evaluated.” 


3 


Table  2.  Air  Force  MDC  Related  Systems  Reviewed 
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and  availability  tracking  and  bas  been  operational  for  several  years.  A  new  system  tbat  will  include  transparent  data 

management  for  tactical  missiles.  collection  as  well  as  other  features  is  now  in  the  planning  stages. 


Table  3.  Navy  MDC  Related  Systems  Reviewed 
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that  will  permit  transparent  data  collection  by  using  bar  code  readers. 


Table  4.  Army  MDC  Related  Systems  Reviewed 


23  METHOD 


Acknowledging  that  DoD  MDC  systems  are  not  intended  to  improve  diagnostics 
capabilities,  we  attempted  to  assess  and  characterize  the  application  of  DoD  MDC  systems  to 
support  comparable  levels  of  diagnostics  improvement  found  in  the  commercial  case  studies.  The 
conclusions  and  recommendations  presented  in  this  study  are  based  on  the  findings  resulting  from 
personal  interviews,  MDC  program  documentation,  site  visits,  and  the  results  of  the  earlier 
Integrated  Diagnostics  Workshop  Study. 

Discussions  with  the  various  MDC  program  offices,  focused  on  the  type  and  range  of  data 
collected,  the  intended  purpose  of  this  information,  the  identification  of  examples  or  initiatives 
where  the  data  are  intended  for  diagnostics  related  efforts,  comparisons  of  DoD  MDC  system 
attributes  with  those  identified  during  the  Integrated  Diagnostics  Workshop,  and  opportunities  to 
improve  feedback  and  analyses  capabilities  through  enhancement  of  MDC  systems. 

During  visits  with  maintenance  personnel,  discussions  focused  on  how  the  data  are 
collected,  what  portions  of  the  data  colleaed  are  used  at  the  local  maintenance  level,  and  how 
frequently  the  data  are  used  to  support  diagnostic  related  needs.  Discussions  also  addressed  current 
and  future  technical  information  needs.  This  discussion  topic  was  included  due  to  the  high  degree 
of  data  correlation  found  in  the  commercial  sector  examples  between  technical  manuals,  the  system 
configuration,  the  appropriate  diagnostic  procedures,  and  the  pertinent  maintenance  information 
collected. 

2.4  STRENGTHS  AND  WEAKNESSES  OF  APPROACH 

Although  the  research  is  intended  to  apply  to  all  DoD.  systems,  this  study  tends  to 
emphasize  aviation  maintenance  applications.  The  vast  number  of  MDC-related  systems  made  it 
impractical  to  look  at  the  full  range  of  applications;  and  additionally,  the  primary  DoD  MDC 
systems  available  for  study  tended  to  be  associated  with  this  application  area. 

This  study  is  a  direct  result  of  the  following  observations:  (1)  Commercial  systems  with 
highly  effective  diagnostics  capabilities  typically  employ  effective  MDC  that  rapidly  feeds  back 
diagnostic  improvement  information  to  the  design  and  maintenance  communities,  and  (2)  DoD 
MDC  systems,  although  they  collea  huge  amounts  of  information,  are  not  directly  tied  to 
diagnostics  improvement  processes.  It  is  natural  to  ask  whether  the  commercial  diagnostic 
improvements  were  due  to  the  effective  MDC  systems  or  if  the  improvements  might  have  been  due 
in  part  to  other  factors  such  as  increased  management  awareness.  Although  full  consideration  of  this 
issue  is  beyond  the  scope  of  this  study,  we  believe  that  concerns  regarding  this  reasoning  are 
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minimized  by  the  methods  used  in  this  study.  Specifically,  the  study  focused  on  the  opportunities 
to  use  the  collected  information  to  improve  diagnostics  capabilities  on  operational  weapon 
systems,  independent  of  whether  efiFective  MDC  applications  were  the  cause  of  diagnostics 
in^rovements  for  the  commercial  systems  reviewed. 

Exceptions  to  the  general  MDC  system  attributes  and  application  trends  identified  in  this 
study  may  be  found  throughout  both  the  commercial  sector  and  DoD.  We  believe  that  the 
significance  of  observations  in  this  study  rests  not  with  the  identification  of  specific  weaknesses 
for  any  commercial  or  DoD  MDC  applications,  but  rather,  with  the  identification  of  opportunities 
to  enhance  maintenance  and  diagnostics  performance  across  a  broad  range  of  systems. 


3.  OBSERVATIONS 


The  following  present  the  study  observations.  These  observations  illustrate  many  of  the 
underlying  causes  of  diagnostic  accuracy  and  maintenance  efficiency  problems  and  help  identify 
opportunities  to  improve  diagnostic  capabilities  through  effective  feedback  of  MDC  information. 
Since  many  of  these  observations  apply  equally  across  functional  areas,  they  have  been  grouped 
under  four  convenient  topics  that  provide  a  composite  view  of  DoD  maintenance  and  diagnostic 
environments;  factors  influencing  diagnostic  accuracy  and  maintenance  efficiency,  specific  MDC 
system  deficiencies,  MDC  system  costs,  and  comparison  of  observed  DoD  practices  with 
commercial  MDC  implementations.  All  interviews  or  discussions  noted  in  the  following  sections 
were  conducted  as  a  part  of  this  study  unless  a  reference  is  provided  at  the  end  of  the  paragraph. 


3.1  FACTORS  INFLUENCING  DIAGNOSTIC  ACCURACY  AND  MAINTENANCE 
EFFICIENCY 

A  wide  range  of  factors  were  observed  to  influence  DoD’s  diagnostic  accuracy  and 
maintenance  efficiency.  The  following  list  summarizes  the  observations  presented  in  this  section. 

a.  DoD  lacks  dedicated  organizations  and  processes  for  improving  diagnostics. 

b.  MDC  systems  are  slow  responding  to  maintenance  technician  needs. 

c.  The  experience  of  the  workforce  is  limited  relative  to  the  required  maintenance  tasks. 

d.  Technical  manuals  contain  inaccurate  diagnostic  procedures. 

e.  Technical  manuals  are  rarely  followed  precisely. 

f.  Information  that  should  be  stored  in  MDC  systems  is  frequently  false  or  missing. 

g.  MDC  systems  generally  increase  the  maintenance  technician  work  load. 

h.  Data  entry  processes  are  constant  sources  of  error. 

i.  MDC  data  are  incomplete  and  inaccurate  relative  to  diagnostic  improvement  needs. 
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3.1.1  Deficiency  Reporting 


a.  We  found  no  Service  organizations  that  acknowledged  responsibility  for  the  purposeful 
use  of  MDC  data  to  improve  diagnostic  capabilities  as  a  primary  mission. 

b.  Maintenance  personnel  fi'om  the  Anny,  Navy,  and  Air  Force  observed  that  the  primary 
approaches  used  by  the  Services  to  identify  and  report  weapon  system  deficiencies  are 
slow,  non-responsive,  and  not  integrated  with  the  MDC  systems.  They  indicated  that 
the  various  deficiency  reporting  (DR)  mechanisms  used  by  the  Services  are  based  on 
infrastructures  that  are  fragmented  across  product  and  support  levels,  and  depend  on 
data  from  sources  that  are  outside  the  DR  span  of  control.  They  noted  that  rarely  is 
MDC  data  used  to  support  or  improve  diagnostic  performance. 

c.  The  Navy  lETM  study  found  that  the  Technical  Publication  Deficiency  Reporting 
(TPDR)  process  is  not  responsive  to  corrective  needs  and  that  less  than  50%  of  those 
interviewed  ever  receive  confirmation  of  disposition  for  submitted  reports.  Another 
approximately  25%  never  submitted  a  TPDR;  this  high  percentage  is  attributed  to  the 
low  level  of  confidence  in  the  reporting  process  [ETM  1990]. 

d.  The  Air  Force  has  formal  processes  whereby  maintenance  technicians  may  report 
deficiencies  for  both  the  design  (using  Material  Deficiency  Reports  or  MDRs)  and  the 
technical  manuals  called  Techm'cal  Orders  (using  AFTO  Form  22).  In  both  cases  the 
technicians  feel  that  a  great  deal  of  time  elapses  between  submitting  a  report  and 
receiving  a  response,  and  the  quality  of  the  response  by  the  evaluator  is  generally  quite 
low.  Air  Force  personnel  who  are  assigned  the  responsibility  of  answering  these  reports 
acknowledge  that  they  do  not  have  adequate  resources  to  either  review  the  reports  or, 
more  importantly,  to  initiate  corrective  actions.  The  problems  associated  with 
deficiency  reporting  for  AF  Technical  Orders  (TO)  represent  “an  especially  serious 
situation  because  TOs  are  inevitably  immature  when  initially  fielded  and  consequently 
need  extensive  field  participation  for  their  inq>rovement  [Gebman,  1988].” 


3.1.2  MDC  Systems  Slow  Responsiveness  to  Operational  Maintenance  Needs 

a.  Persormel  interviewed  stated  that  the  Navy  Data  Storage  Units  (DSU)  on  board  some 
of  the  aircraft  are  not  read  until  the  capacity  of  the  unit  is  around  80%  full  (approxi¬ 
mately  3  flights).  This  practice  varies  from  squadron  to  squadron,  and  the  use  of  the 
information  appears  to  be  driven  by  personalities  more  than  by  policy.  However,  the 
data  from  the  DSUs  are  down-loaded  more  fi^equently  if  mission  critical  information  is 
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needed  by  the  mission  planners  or  operations  personnel. 

b.  Navy  maintenance  personnel  experienced  in  aircraft  carrier  operations  observed  that 
current  diagnostic-related  MDC  data  are  not  available  in  time  to  support  the  normal 
tum-around  times  dictated  by  operational  scenarios. 

c.  Air  Force  maintenance  personnel  indicated  that  it  is  common  for  an  aircraft  grounding 
problem  to  be  diagnosed  and  fixed  before  the  data  is  entered  into  the  CAMS  data  base. 
This  also  was  witnessed  by  the  study  team  during  site  visits. 


3.13  Limited  Workforce  Experience 

a.  Fifty  percent  of  the  personnel  interviewed  in  a  Navy  lETM  study  had  only  1  to  3  years 
in  their  current  work  center.  An  additional  34%  had  less  than  1  year  in  the  current  work 
center  [lETM  1990]. 

b.  Fifty-seven  percent  of  the  personnel  interviewed  in  a  Navy  lETM  study  had  1  to  3 
years  experience  on  their  current  weapon  system.  An  additional  19%  had  less  than  1 
year  on  their  current  weapon  system  [lETM  1990]. 

c.  Contractors  for  both  the  F-15  and  F-16  radars  found  that  debriefing  of  pilots  is 
commonly  handled  by  maintenance  technicians  who  lack  specialized  knowledge  about 
the  radar  systems.  To  compound  the  situation,  these  technicians  often  must  debrief 
multiple  sorties  by  several  pilots  at  one  time.  “Because  of  the  shortcomings  of  these 
debriefings,  pilots  fail  to  provide  information  necessary  to  identify  faults  and  fault 
codes,  and  maintenance  personnel  often  ignore  the  radar  malfunction  codes  produced 
at  the  debriefing.”  In  addition,  the  debriefers  who  are  unfamiliar  with  the  radar  systems 
experience  great  difficulty  using  the  fault  code  manual.  “During  the  radar  R&M 
program,  debriefing  personnel  correctly  assigned  the  proper  fault  code  only  35%  of  the 
time  [Gebman  1988].” 

d.  During  a  pilot  debriefing,  witnessed  as  a  part  of  this  study,  the  technician  performing 
the  debriefing  was  unable  to  locate  the  proper  code  in  the  aircraft  Fault  Reporting 
Manual.  Furthermore,  this  individual  had  very  limited  maintenance  and  systems 
experience.  Discussions  with  the  Dispatcher  (the  individual  who  usually  assigns  the 
tasks  to  a  specific  shop  based  on  the  results  of  the  debriefing  process)  indicated  that  he 
(the  dispatcher)  did  not  place  much  credibility  in  the  fault  codes  coming  from  the  pilot 
debriefing  process  and  simply  attempted  to  send  the  task  to  the  appropriate  shop  for 
action. 


3.1.4  Technical  Instructions  Not  Followed  and  Contain  Immature  Diagnostics 

a.  The  Navy  lETM  study  found  that  due  to  operationally-related  environmental  con¬ 
straints,  the  Navy  technical  manuals  (including  the  check  lists)  are  used  less  than  50% 
of  the  time  in  a  step-by-step  manner  at  cither  the  intermediate  or  organizational  (I  or 
O)  level  of  maintenance  [lETM  1990]. 

b.  Research  under  the  lETM-related  studies  demonstrated  an  increased  acceptance  by 
maintenance  personnel  to  use  available  technical  information  when  the  information 
was  provided  by  electronic  technical  manuals.  These  studies  also  demonstrated 
substantial  opportunities  to  improve  diagnostics  accuracy  (see  Section  B.2.5,  “lETM”, 
for  additional  information). 

c.  During  an  Air  Force  R&M  Study,  maintenance  techniciaris  were  seldom  observed 
referring  to  technical  manuals  except  when  dealing  with  the  most  difficult  faults,  such 
as  wiring  problems  [Gebman  1988]. 

d.  Air  Force  Tactical  Air  Command  (TAC)  discovered  that  the  F-15  trouble-shooting 
Technical  Order  was  not  being  followed  because  the  manuals  were  too  complicated 
and  many  of  the  fault  codes  were  inaccurate.  Upon  further  study,  TAC  discovered 
approximately  350  voids  in  the  fault  tree  documented  by  these  manuals  (see  Section 
B.1.5,  “CFR”  for  additional  information). 

3.U  Environmental  and  Operational  Constraints 

a.  Maintenance  personnel  interviewed  during  this  study  indicated  that  the  physical  use 
and  step-by-step  application  of  fault  isolation  and  detection  information  from  the  tech¬ 
nical  manuals  and  even  check  lists  are  often  severely  hampered  by  environmental  con¬ 
ditions  (bad  weather,  night  time  conditions,  flight-deck  maintenance  environments, 
mud,  etc.). 

b.  Rarely  is  it  practical  to  bring  all  of  the  needed  technical  information  to  a  problem  site. 
A  trace  through  the  reference  material  for  a  moderately  complex  task  on  the  F-18 
aircraft  revealed  tliat  11  manuals  and  33  work  packages  were  required  to  perform  the 
identified  task  from  fault  isolation  through  final  checkout  [IETM1990]. 

c.  We  observed  that  the  DoD  MDC  systems  are  neither  convenient  for  the  technician  nor 
compatible  with  the  maintenance  process.  For  example,  the  Air  Force  CAMS 
terminals,  as  currently  implemented,  are  placed  at  locations  separate  from  the 
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organizational  maintenance  work  sites,  thus  making  CAMS  ineffective  for  immediate 
diagnostic  analyses.  EXiring  the  visits  maintenance  personnel  indicated  that  the  lack  of 
CAMS  equipment  at  maintenance  sites  frequently  forces  the  technician  either  to  remember 
the  maintenance  data  for  future  entry  into  CAMS  or  to  generate  paper  documentation  that 
must  later  be  entered.  Both  of  these  situations  affect  the  maintenance  workload  and  are 
prone  to  human  error. 

3.1.6  MDC  Information  Integrity 

a.  During  site  visits,  one  maintenance  technician  cited  a  case  where  the  CAMS  infonnation 
showed  4  sets  of  different  serial  numbers  for  a  single  LRU  part  number  installed  in  a  F- 1 5 
radar  even  though  the  aircraft  radar  will  accept  only  one  of  these  units. 

b.  A  pilot  debriefing  session  was  witnessed  where  no  maintenance  actions  were  recorded  in 
the  MDC  data  (i.e.,  the  airaaft  was  reported  to  have  landed  in  a  category  1  status). 
However,  in  fact,  the  aircraft  had  landed  with  a  grounding  condition  (category  3)  due  to  a 
radar  problem.  In  this  specific  case  a  serialized  unit  was  removed  and  replaced  by 
maintenance  prior  to  the  pilot  debriefing.  Maintenance  personnel  observed  that  in  order  to 
keep  an  organization’s  quality/performance  metrics  high,  the  maintenance  actions  of  this 
nature  are  frequently  not  reported.  As  a  direct  result  of  these  events,  the  MDC  data  for  that 
radar  now  had  an  invalid  LRU  serial  number  and  the  true  “how  malfunction”  and 
“discrepancy”  codes  for  the  failed  unit  were  suspect 

c.  Both  Air  Force  and  Navy  maintenance  personnel  cited  examples  where  the  technicians 
select  convenient  or  easy  to  remember  codes  for  many  of  the  MDC  data  fields  rather  than 
looking  up  the  precise  code  for  the  specific  condition.  The  people  interviewed  attributed 
this  problem  to  the  proliferation  of  potential  codes,  the  time  consuming  problem  of  looking 
up  specific  codes,  and  the  level  of  training  and  experience  of  the  work  force. 

d.  Personnel  from  an  Air  Force  avionics  maintenance  shop  cited  examples  where  they  have 
entered  the  data  and  later  learned  that  due  to  a  subsequent  CAMS  down  time  or  lock-up, 
the  data  was  lost.  However,  since  the  data  was  entered  in  near  real  time  (the  way  the  system 
is  intended  to  work),  paper  back  up  information  had  not  been  created.  They  commented 
that  CAMS  does  have  a  system  save-to-tape  capability;  however,  it  does  not  always 
prevent  the  loss  of  data. 


3.1.7  Work  Load  Impact 

a.  We  observed  that  the  improvements  offered  by  the  MDC  systems  studied  aio  princi¬ 
pally  those  of  automating  paper  based  functions  and  providing  interconnections 
between  various  existing  logistics  and  MDC  data  bases.  The  Service  MDC  systems 
(both  paper-based  and  automated)  are  designed  to  collea  iitformation  for  the  manage¬ 
ment  of  maintenance  activities  and  to  provide  parts/equipment  usage  information.  In 
general,  the  workload  of  key  punching  information  from  pap  r-based  maintenance 
activity  summary  forms  was  reduced  by  the  new  MDC  systems.  However,  a  new,  addi¬ 
tional,  data  entry  workload  was  imposed  on  the  skilled  maintenance  technician. 

b.  Air  Force  aircraft  crew  chiefs  cited  numerous  examples  where  the  CAMS  data  entry 
and  forms  checks  took  more  time  to  complete  than  the  actual  maintenance  action 
required  to  fix  an  aircraft.  One  crew  chief  cited  a  specific  example  where  the 
maintenance  actions  on  his  aircraft  (which  had  been  in  a  cannibalized  status  for  a 
period  of  time)  took  approximately  one  and  a  half  days  to  return  the  aircraft  to  a  flight 
worthy  condition.  However,  the  CAMS  data  input  and  job  close  out  took  three  days. 

c.  Most  of  the  crew  chiefs  interviewed  felt  that  the  data  entry  in  CAMS  represented  a 
duplication  of  the  required  aircraft  paper  forms.  Without  exception,  everyone 
acknowledged  that  the  CAMS  information  l-f^geti  oehind  the  data  entry  in  the  required 
paper  forms,  and  that  the  various  paper  forms  were  the  primary  means  of  determining 
immediate  update  suitus.  There  was  a  perception  that  paper  would  always  be  the 
primary  source  of  immediate  surv.s  informati'^n  due  to  the  many  potential  problems 
that  could  influence  the  CAMS  output  (power  outage,  deployment,  remote  site 
operations,  down  time,  etc.). 

d.  We  observed  that  the  DoD  MDC  systems  lack  common  communications  links  that 
permit  convenient  data  transfer  between  MDC  related  systems  as  well  as  transparent 
data  collection  from  operational  weapon  systems.  In  general,  we  found  that  DoD  uses 
manual  communication  links  for  entering  data  into  the  MDC  systems.  Even  when  the 
data  are  captured  automatically,  the  information  finds  its  way  back  into  paper  forms 
that  must  be  later  key  punched  into  another  MDC  system. 


3.1.8  Data  Entry  Concerns 

a.  The  CAMS  users  felt  that  additional  internal  editing  and  processing  could  make  the 
system  more  responsive  to  their  needs.  Examples  they  cited  included  more  checking  of 
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input  fields  to  ensure  data  compatibility  at  the  time  of  input,  and  more  automatic  data 
entry  of  common  data  on  subsequent  screens. 

b.  All  of  the  CAMS  users  interviewed  felt  that  there  are  too  many  different  screens  to 
remember.  They  also  commented  that  the  response  time  of  some  screens  is  excessively 
slow.  Furthermore,  in  the  menu  mode  (CAMS  provides  screen  prompts),  the 
responsiveness  of  the  system  is  even  worse. 

c.  CAMS  data  entry  proficiency  was  identified  as  a  significant  concern  in  each  of  the 
maintenance  areas  visited.  The  users  commented  that  this  problem  is  compounded  by  the 
constant  software  updates.  Limited  training  is  available  for  CAMS  and  most  of  the 
personnel  gain  proficiency  through  on-the-job-training.  Fortunately,  this  training  is  made 
manageable  by  the  fact  that  each  maintenance  area  needs  to  learn  only  a  few  of  the 
CAMS  screens.  Approximately,  one-quarter  of  the  operators  observed  during  the  visit 
requested  assistance  for  selecting  the  appropriate  code,  screen  number,  mode,  or  input 
data.  A  common  practice  in  some  of  the  maintenance  areas  was  to  designate  an  expert 
CAMS  user  as  the  individual  responsible  for  entering  data. 

d.  On  two  occasions  during  the  site  visit  to  a  CAMS  user,  the  study  team  witnessed 
maintenance  personnel,  who  were  entering  data  into  CAMS,  being  reassigned  to  other 
maintenance  tasks.  Personnel  interviewed  indicated  that  this  is  a  commc.*  problem.  Data 
entry  is  assigned  a  lower  priority  than  the  maintenance  actions.  Maintenance  personnel 
observed  that  since  paper-forms  have  been  filled  out  at  the  aircraft  (thus  meeting  basic 
safety  and  operations  concerns),  there  exists  a  general  policy  that  the  CAMS  entry  may 
be  deferred  until  later. 


3.1.9  Incomplete  and  Inaccurate  MDC  Data 

a.  Figure  1  summarizes  the  diagnostics-related  information  resulting  from  maintenance 
actions.  It  highlights  critical  information  that  is  typically  not  reported  in  the  DoD  MDC 
systems,  yet  is  essential  for  diagnostics  process  and  design  evolution.  Without  this  infor¬ 
mation  it  is  not  practical  to  correlate  the  reported  problems,  the  diagnostic  procedures 
applied,  the  equipment  configuration,  and  the  corrective  maintenance  actions. 
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Figure  1.  Diagnostics  Related  Intbrmation  Resulting  from  Maintenance  Actions 


How  Mai  codes  address  both  end  item 
as  well  as  sub-assemblies  or  components. 
Includes  codes  for  condition-not-found, 
CND/RTOK,  etc. 


b.  A  recent  Rand  Corporation  report  noted  that  the  following  support  weaknesses  contribute  to 
poor  fault  isolation  performance:  inadequate  information  about  avionics  equipment  during 
flight,  the  lack  of  serial  niunber  tracking  of  avionics  equipment,  and  inadequate  integration  of 
maintenance  efforts  and  fault  information  across  maintenance  levels  [Gebman  1989,  p.32]. 
We  observed  that  the  contributing  factors  to  each  of  these  weaknesses  would  be  reduced  by 
instituting  processes  that  collect  the  missing  information  highlighted  in  Figure  1. 

c.  There  was  general  agreement  among  the  personnel  interviewed  that  the  automated  MDC 
system  improved  the  accuracy  of  the  collected  information.  However,  they  also  identified 
persistent  problems  involving  the  integrity  of  the  collected  data  that  were  not  eliminated.  Error 
sources  most  frequently  cited  included  errors  based  on  the  technician  remembering  the  proper 
codes,  personnel  selecting  convenient  codes  which  will  always  be  accepted  by  the  system, 
multiple  serial  number  entries  for  the  same  part  number,  re-key  punching  data  from  paper, 
proficiency  of  personnel  using  the  system,  the  long  delays  from  the  time  the  data  was  collected 
until  the  time  that  the  data  is  entered  into  the  system,  incompatibility  of  the  systems  to  accept 
the  data  in  the  same  time  sequence  as  it  was  acquired,  lost  data  due  to  system  crashes,  frequent 
work-around-tasks  that  are  not  entered  into  the  MDC  data  in  order  to  inflate  reported 
organizational  metrics,  and  the  unfriendly  nature  of  some  of  the  MDC  systems. 


3,2  SPECIFIC  MDC  SYSTEM  DEFICIENCIES 

The  following  sections  summarize  MDC  system  specific  deficiencies  observed  by  the  authors 
during  this  study. 


3,2.1  CASS  and  NALCOMIS  Information  Interface 

CASS  will  incorporate  several  desirable  features  found  in  the  commercial  sector  MDC  systems: 
automated  (transparent)  data  collection,  paperless  technical  manuals  resident  in  the  system,  and 
automated  storage  and  retrieval  of  maintenance  information  within  the  CASS.  It  also  will  automatically 
produce  the  VIDS/MAF  (paper)  forms  used  in  the  Naval  Aviation  Maintenance  Program  (NAMP)  for 
both  maintenance  production  control  and  maintenance  data  collection. 

However,  the  direct  electronic  link  to  NALCOMIS  and  other  MDC  type  systems  was  eliminated 
due  to  budget  and  program  constraints.  At  present,  the  information  found  on  this  VIDS/MAF  form  will 
have  to  be  re-entered  manually  into  NALCOMIS.  Furthermore,  the  additional  detailed  diagnostic-related 
information  which  will  be  collected  by  CASS  and  will  be  essential  for  correlation  of  maintenance 
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processes,  is  not  included  on  the  VIDS/MAF  form.  Therefore  a  significant  portion  of  the  detailed 
MDC  information  will  not  be  available  for  centralized  diagnostic  analyses. 


^22  CAMS  Performance  and  Operational  Inefficiencies 

Although  CAMS  provided  improvement  in  both  MDC  efficiency  and  data  integrity  over  the 
past  seven  years,  significant  system  performance  and  operational  inefficiencies  persist.  The  current 
CAMS  problems  are  related  to  several  very  significant  conditions:  (I)  the  system  is  based  on  over  ten 
year  old  technology,  (2)  the  system  is  not  hosted  on  its  own  dedicated  computer,  (3)  the 
implementation  concept  is  focused  on  automating  a  paper  MDC  concept  as  opposed  to  improving 
maintenance  efficiency  and  (4)  the  system  is  not  accessible  at  the  on-equipment  maintenance  sites. 
The  system  and  operadonal  problems  involving  the  CAMS  are  summarized: 

•Excessive  system  down  time 
•Too  many  screens  and  slow  responsiveness  of  screens 
•Work  load  impact  and  excessive  time  required  for  data  entry 
•Persistent  training  and  proficiency-related  problems 
•Lags  as  well  as  duplicates  the  aircraft  paper  forms 
As  a  consequence  of  these  problems,  CAMS  provides  very  little  support  to  the  maintenance 
technician,  and  firequently  represents  an  additional  work  load  burden.  In  faa,  very  few  of  the  CAMS 
benefits  apply  directly  to  the  maintainer. 

3^  IVirbine  Engine  Prognostic  Capabilities  (CEMS IV  and  ECAMS) 

A  significant  number  of  Air  Force  and  Navy  turbine  engines  are  compatible  with  the 
advanced  monitoring  capabilities:  however,  only  a  fraction  of  these  engine  types  have  been  analyzed 
sufficiently  to  assure  accurate  maintenance  prognostic  capabilities.  CEMS  IV  provides  a  very 
comprehensive  capability  to  support  engine  diagnostics  and  to  prognosticate  trends  that  will  require 
maintenance  actions.  The  system  will  collect  parametric  engine  performance  data  transparently 
(automatically,  and  without  additional  maintenance  technician  intervention)  and  couple  this 
information  with  oil  analysis  program  (OAP),  component  life  expenditures,  and  other  maintenance 
history  data  already  collected.  However,  at  present  within  the  Air  Force,  only  the  technical  data  for 
the  A- 10  aircraft  engine  (TF-34)  has  been  sufficiently  analyzed  to  yield  the  full  range  of  prognostic 
and  diagnostic  opportunities  ftom  the  CEMS  IV. 
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Currently,  similar  parametric  information  is  being  collected  for  the  B-1,  F- 15, and  F-16 
engines,  but  the  engine  technicians  are  not  able  to  use  this  diagnostics  capability  to  its  fullest  extent 
since  they  are  not  always  certain  what  the  trend  data  they  are  collecting  actually  indicates. 

The  Navy  ECAMS  system  automatically  down  loads  engine  and  other  aircraft  parametric 
data  via  a  data  storage  unit  (DSU).  As  a  result  of  situations  similar  to  those  identified  for  CEMS 
IV,  ECAMS  currently  supports  trend  and  event  diagnostics  only  on  the  new  upgrades  to  the  F- 14 
aircraft  engines  (F-110).  ECAMS  does  not  provide  the  same  capability  for  the  F-18  aircraft 
engines  (F-404). 

The  problem  associated  with  the  lack  of  analyzed  information  necessary  to  support  the  full 
range  of  prognostic  opportunities  is  further  exacerbated  by  the  fact  that  there  exists  two  distinctly 
different  software  implementation  of  ECAMS  for  the  F-18  and  F-14  aircraft.  The  Navy  is 
investigating  opportunities  to  develop  a  single  ECAMS  implementation. 


3.2.4  Limited  SAMS  Codified  Data. 

Although,  SAMS  is  providing  a  greater  continuous  flow  of  maintenance  information  for 
centralized  analyses,  the  information  collected  is  at  too  high  a  level  of  abstraction  to  support 
diagnostics  improvement.  Specific  detailed  codified  information  that  addresses  the  reported 
problem,  parametric  operational  data,  the  serial  number  of  involved  parts,  and  actions  taken  are  all 
essential  ingredients  for  the  Army  MDC  system  to  effectively  support  diagnostics  improvement. 


33  MDC  SYSTEM  COSTS 

The  cost  to  develop  and  field  the  MDC  system  features  reviewed  in  this  study  are  in  the 
order  of  several  billion  dollars.  A  rigorous  assessment  of  these  costs  is  beyond  the  scope  of  this 
study.  Furthermore,  it  would  be  very  difficult,  and  highly  subjective,  to  separate  the  specific  MDC 
related  development  costs  from  the  cost  of  other  functional  capabilities  in  many  of  the  systems 
studied.Therefore,  the  following  cost  information  provided  by  the  respective  program  offices 
provides  only  a  top  level  review  of  the  costs  associated  with  selected  MDC  systems.  It  should  also 
be  noted  that  the  various  cost  amounts  address  a  range  of  then-year-dollars  and  represent  a  mix  of 
development,  acquisition,  and  operations  and  maintenance  (O&M)  resources. 

We  observed  firom  the  various  MDC  system  program  documentation  that  the  justification 
and  rationale  for  the  individual  programs  focused  on  areas  other  than  diagnostics  accuracy 
improvement.  We  inferred  from  this  that  the  synergistic  processes  used  to  improve  diagnostic 
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performance,  similar  to  those  found  in  the  commercial  case  studies  during  the  Integrated 
Diagnostics  Woricshop  [Brown  1990],  are  not  funded  goals  of  the  following  programs. 

a.  CAMS:  Estimated  development,  acquisition,  and  O&M  costs  covering  approximately 
10  years  through  to  full-operational-capability  (FOC)  projected  for  November  1991  - 
$265  million. 

b.  NALCOMIS:  Estimated  life  cycle  costs  that  includes  development,  acquisition  and 
support  for  30  years  --  $1.3  billion. 

c.  REMIS:  Estimated  development,  acquisition  and  O&M  costs  covering  approximately 
10  years  through  FOC  project  for  FY  95  --  $136  million. 

d.  DMMIS:  Estimated  development,  acquisition  and  O&M  costs  covering  approximately 
6  years  through  FOC  projected  FY  93  —  $242  million. 

e.  CEMS:  Estimated  acquisition  cost  for  the  turbine  engine  monitoring  system  (TEMS) 
used  to  collect  the  parametric  data  from  the  A- 10  aircraft  engine  for  CEMS  -  $94 
million.  The  estimated  development  and  acquisition  costs  for  the  CEMS  data  storage 
and  retrieval  capabilities  --  $35  million. 


3.4  COMPARISON  OF  OBSERVED  DOD  PRACTICES  WITH  COMMERCIAL  MDC 
IMPLEMENTATIONS 


The  following  compare  the  observed  DoD  maintenance  and  diagnostic  practices  with  the 
commercial  MDC  implementations. 

a.  We  observed  that  current  DoD  MDC  systems  may  highlight  diagnostic  problems  but 
do  not  pinpoint  the  diagnostic  improvements  needed.  This  dramatically  contrasts  with 
commercial  case  studies  previously  evaluated  at  the  Integrated  Diagnostics  Workshop 
in  which  the  commercial  systems  were  found  to  specifically  emphasize  identification 
and  resolution  of  diagnostic  and  equipment  problems  [Brown,  1990]. 

b.  Based  on  the  general  information  gathered  during  the  study,  the  authors  perceive  that 
DoD  tends  to  place  higher  priority  on  availability  than  costs.  This  trend  was  found  to 
contrast  with  the  commercial  sector  case  studies,  where  costs  were  assigned  higher 
priority.  This  led  the  authors  to  observe  that  DoD  tended  to  automate  the  processes  of 
MDC  for  the  purpose  of  enhancing  maintenance  logistics  and  resource  utilization, 
whereas,  the  commercial  sector  tended  to  automate  diagnostics  maintenance  aids  for 
the  purpose  of  providing  rapid  in^rovement  of  diagnostic  and  product  designs. 

c.  The  ability  to  identify  and  isolate  opportunities  for  improving  diagnostic  and 
maintenance  efficiency  within  the  DoD  environment  were  found  to  be  highly 
personality  dependent.  Rarely  was  an  individual  or  organization  observed  to  be 
responsible  for  specifically  improving  the  accuracy  and  capability  of  the  individual 
diagnostic  elements.  In  contrast,  the  commercial  cases  reviewed  specifically  assigned 
the  responsibility  of  diagnostic  inprovement  to  an  organization. 

d.  Table  5  compares  the  commercial  and  DoD  MDC  implementation  practices  observed 
during  this  study. 

e.  Table  6  compares  the  general  commercial  and  DoD  MDC  system  application  scopes 
observed  during  this  study. 


Table  5.  Comparison  of  Commercial  and  DoD  MDC  Attributes 
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(1)  Conelatiog  tbe  maintenance  process  refers  to  the  ability  to  thread  from  the  reported  problem,  to  the  diagnosis,  to  the 
maintenance  actions,  and  Anally,  to  the  results  of  the  maintenance  acAons. 


Capabilities  Air  Force  Navy  Army  Commercia 


\ 


Table  6.  Comparison  of  Commercial  and  DoD  MDC  System  Scope 


(1)  Includes  both  paper  based  and  automated  DoD  MDC  information. 


4.  CONCLUSIONS 


4.1  DOD  LACKS  A  FOCUSED  PROCESS  FOR  IMPROVING  DIAGNOSTICS 

DoD’s  diagnostic  capabilities  are  limited  by  the  lack  of  a  focused  process  for  improving 
diagnostic  performance.  Slow  diagnostic  accuracy  improvement  is  a  systemic  DoD  problem.  The 
maintenance  and  support  processes,  in  contrast  with  the  commercial  cases  studied,  rarely  use  the 
MDC  information  to  identify  opportunities  for  diagnostic  improvement.  As  a  result,  the 
maintenance  and  support  processes  used  by  weapon  system  managers  tend  to  perpetuate  the  status 
quo  rather  than  improve  diagnostic  capabilities.  Many  of  the  diagnostic  inaccuracies  that  are 
present  at  the  time  a  new  system  is  fielded  (or  at  the  time  a  system  modification  is  implemented) 
have  a  tendency  to  re'  for  the  life  of  the  system.  Consequently,  the  cost  avoidance 
opportunities  that  rr  xy  attributed  to  more  accurate  diagnostic  and  enhanced  maintenance 
efficiency  are  los^ 


4.1,1  Diagnostic  Process  Systemic  Probiems 

We  conclude  that  the  DoD  diagnostic  improvement  process  lacks  three  basic  capabilities: 

a.  The  existing  mjuntenance  approach  does  not  effectively  identify  the  source  or  cause  of 
diagnostic  problems. 

b.  When  diagnostic  problems  are  identified,  the  maintenance  and  support  approach  does 
not  effectively  report  the  diagnostic  problems. 

c.  When  problems  are  reported,  the  Services  lack  individuals  or  organizations  assigned 
the  primary  responsibility  to  analyze  and  fix  the  cause  of  the  diagnostic  inaccuracy. 

4.12  Continuous  Feedback  of  Relevant  Technical  Information  Lacking 

The  diagnostic  and  design  improvement  processes  lack  continuous  feedback  of  relevant 
technical  information.  In  the  absence  of  such  feedback,  many  problems  remain  invisible.  In  order 


for  DoD  to  address  a  diagnostic  inaccuracy  problem,  a  systems  feedback  and  analysis  approach  is 
needed  that  reports  the  problem  source,  identifies  corrective  actions,  and  provides  justification  to 
expend  corrective  action  resources. 

DoD  MDC  systems  do  a  reasonable  job  of  tracking  reliability,  maintainability,  and 
availability.  However,  they  are  ineffective  at  identifying  diagnostic  inaccuracy  problems.  The 
available  MDC  information  is  abstracted  to  such  a  high  level  that  the  data  lack  relevant  technical 
information  necessary  to  determine  if  the  reported  factors  are  at  desirable  levels  or  they  simply 
represent  the  original  immature  diagnostic  performance  delivered  with  the  design. 

4.13  Organizations  Accountable  for  Diagnostic  Improvement  Lacking 

The  responsibility  and  authority  for  identifying  diagnostic  inaccuracies,  developing 
diagnostic  improvements,  and  implementing  corrective  actions  reside  at  different  organizations  for 
each  of  the  diagnostic  elements.  Each  of  these  organizations  has  other  responsibilities  that  focus  on 
immediate  problem  solutions.  Diagnostic  inaccuracy  problems,  if  identified,  tend  to  be  ignored  since 
fimding  is  usually  directed  at  solving  immediate  problems,  such  as  buying  spares,  sending  an  item  to 
a  depot,  etc.  However,  the  necessary  resources  to  fix  diagnostic  element  problems  must  come  from 
another  funding  source  that  is  more  difficult  to  access. 

4.1.4  Data  Not  Used  for  Diagnostics 

The  large  amounts  of  maintenance-related  data  available  within  the  Services  are  rarely  used 
to  support  diagnostic  and  system  design  improvement.  These  DoD  MDC  systems  have  been  justified 
based  on  requirements  to  collect  maintenance-related  data  in  support  of  non-diagnostic  specific  areas. 
In  general,  these  systems  meet  or  exceed  the  information  objectives  for  their  intended  areas  of 
support  logistics  planning,  availability  tracking,  reliability  tracking  and  maintenance  production 
management. 

The  individuals  who  receive  this  information  are  rarely  in  functional  oiganizations  that  focus 
on  diagnostic  element  improvements;  and  when  they  are,  they  find  that  the  infoimation  lacks  specific 
data  fields  necessary  to  permit  correlation  across  produrt  and  maintenance  levels. 

4.13  Poor  Data  Entry  Interfaces 

The  point-of-data  entry  interfaces  for  DoD  MDC  systems  are  inefficient,  promote 
inaccuracies,  duplicate  efforts,  and  are  non-responsive  to  the  maintainers’  needs.  In  general  the  DoD 


MDC  systems  are  automating  the  old  paper-based  MDC  concept  and  lack  the  transparent  data 
collection  capabilities  found  in  the  commercial  case  studies.The  lack  of  portable  point-of-data 
entry  interfaces,  that  may  be  taken  to  the  maintenance  site,  are  impacting  work  load,  exacerbating 
data  accuracy  problems,  and  inhibiting  transparent  data  collection  capabilities. 


4.2  THE  FULL  POTENTIAL  OF  MDC  SYSTEMS  ARE  NOT  BEING  REALIZED 

The  DoD  MDC  systems  as  well  as  supporting  automated  data  processing  (ADP) 
capabilities  carry  over  many  of  the  same  problems  as  the  manual  systems  they  are  replacing.  Many 
of  the  normal  benefits  of  automation  are  not  being  fully  realized  because  of  the  continued  need  for 
manpower-intensive  input  and  the  limitations  of  the  hardware  and  software  hosting  the  MDC 
systems. 


4.2.1  DoD  Systems  Not  Targeted  Towards  Diagnostic  Capabilities 

DoD  MDC  systems  are  focused  on  keeping  the  maintenance  logistics  pipeline  filled  by 
automating  the  old  paper-based  maintenance  history  tracking  and  reporting  processes.  The 
Services  use  this  information  for  logistics  planning  and  reporting  maintenance  production.  As 
individual  DoD  MDC  systems  have  evolved,  the  benefits  to  the  maintenance  community  have 
extended  into  areas  such  as  scheduling,  parts  life  tracking,  personnel  and  training  tracking,  etc. 
However,  the  MDC-related  maintenance  workload  is  labor  intensive,  and  the  technician  actually 
performing  the  maintenance  rarely  receives  any  direct  benefit  in  the  diagnostic  process. 


4.22  DoD  MDC  Systems  Do  Not  Support  the  Maintenance  Process 

Even  when  available  MDC  information  could  be  used  to  support  on-equipment 
maintenance,  it  is  rarely  used  due  to  the  time  it  takes  to  get  this  information  out  to  the  maintainer. 
In  contrast,  many  of  the  commercial  MDC  systems  use  portable  maintenance  aids  (PM As)  to 
provide  near  real  time  diagnostic  support  at  the  maintenance  action  site.  These  PMAs  provided 
technical  information,  maintenance  history,  and  diagnostic  procedures. 

The  lack  of  immediately  available  maintenance  history  and  diagnostic  source  iirformation 
at  the  DoD  maintenance  site  creates  situations  where  the  technicians  are  prone  to  remove  the 
wrong  item(s)  for  more  diagnosis  at  an  intermediate  shop  area,  or  to  remove  multiple  items 
knowing  that  one  or  more  of  the  items  are  still  serviceable  (adding  to  the  no-defect-found  rates). 
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4.23  DoD  MDC  Systems  Do  Not  Effectively  Identify  and  Codify  Reported  Problem  Data 


In  general,  the  data  stored  in  the  DoD  MDC  data  bases  lack  the  necessary  information  to 
trace  a  specific  maintenance  action  from  its  initial  diagnosis  through  to  resolution.  Without  this 
information,  correlation  of  needed  corrective  actions  becomes  extremely  difficult  and  often 
impractical. 

43  MDC  PROCESS  IMPROVEMENTS  NEEDED 

Resource  burdens  created  by  diagnostic  inaccuracy  are  in  direct  conflict  with  budget 
reductions  and  the  down-sizing  of  DoD  support  postures.  Commercial  systems  have  demonstrated 
an  ability  to  overcome  and  avoid  these  inaccuracy  problems  by  employing  effective  MDC 
capabilities.  Many  of  these  commercial  MDC  processes  and  technologies  are  available  within  the 
Services;  however,  the  way  the  information  is  collected  and  used  by  the  Services  differs 
significantly. 

We  conclude  that  near  term  diagnostics  improvement  payoffs  are  achievable  by  feeding 
back  existing  diagnostic-related  MDC  information  to  an  individual  or  oiganization  that  has  the 
responsibility  for  improving  diagnostic  accuracy.  However,  improvement  to  diagnostic 
capabilities  will  require  more  than  just  changes  in  the  mechanics  or  scope  of  MDC  systems;  rather, 
fundamental  changes  in  maintenance  capability  and  process  management  will  be  needed.  Systems 
approaches  for  resolving  inaccuracies  will  be  needed  and  we  conclude  that  DoD  MDC  related 
processes  will  require  the  following  basic  improvements. 

a.  Upgrade  MDC  hardware  and  software  to  inaease  the  quality  and  applicability  of  data 
collected,  and  to  increase  MDC  systems  availability. 

b.  Improve  weapon  system  specific  front  end  (point  of  entry)  data  capture  to  increase  data 
accuracy  and  completeness,  to  reduce  ■'-oeralor  work  load,  to  accelerate  the  update  and 
implementation  of  corrective  actions,  and  to  supplement  the  technicians’  knowledge 
with  training  and  current  maintenance  information. 

c.  Adopt  new  analyses  capabilities  to  be  used  by  accountable  organizations  at  both  the 
local  and  central  levels  that  integrate  information  involving  system  configurations, 
maintenance  actions,  maintenance  procedures,  and  identified  discrepancies. 
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5.  RECOMMENDATIONS 


Diagnostic  improvement  and  design  maturation  are  continuing  processes  that  require 
feedback  of  information,  focused  analyses  of  problems  or  inhibiting  factors,  and  feed-forward  of 
corrective  actions.  At  present  within  DoD  these  processes  have  stagnated  and  are  perpetuating  the 
status  quo.  We  have  identified  a  number  of  appropriate  MDC-related  actions  that  should  be  taken 
to  re-enei^ize  and  accelerate  these  processes. 


5.1  ESTABLISH  A  JOINT  SERVICE  DIAGNOSTICS  IMPROVEMENT  TASK  FORCE 

DoD  should  establish  a  Joint  Service  Task  Force  assigned  both  near  and  long  term 
objectives  to  improve  complex  weapon  system  diagnostic  capabilities.  A  near  term  policy  and  long 
term  strategy  for  improving  weapon  system  diagnostic  accuracy  are  needed. 


5.1.1  Near  Term  Accountability  Approach 

The  task  force  should  develop  a  near  term  approach  for  assigning  improvement 
accountability  to  individuals  presently  working  within  each  existing  weapon  system  maintenance 
and  engineering  organization.  The  approach  should  insure  that  these  people  are  assigned  the 
appropriate  maintenance  data  analyses  and  diagnostic  element  improvement  responsibilities.  It 
should  make  full  use  of  existing  ^^DC  capabilities  and  insure  that  these  individuals  are  given  full 
and  timely  access  to  existing  on-  and  off-equipment  diagnostic  parametric  data  as  well  as  all  DR 
information. 


5.12  Long  Term  Road  Map 

The  Task  Force  should  be  asked  to  develop  a  long  term  road  map  for  implementing  a 
diagnostics  improvement  plan.  The  approach  we  suggest  for  DoD  is  outlined  in  our  previous  Joint 
DoDIIndustry  Study  on  Opportunities  in  Integrated  Diagnostics  [Brown  1990]  and  recommends 
that  DoD  develc  p  a  planing  framework  to  facilitate  the  implementation  of  integrated  diagnostic 
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concepts.  This  planning  framework  should  specifically  address  actions  necessary  to  accelerate 
weapon  system  specific  point-of-entry  data  capture  for  increasing  data  accuracy  and  completeness, 
to  extend  common  MDC  and  ATE  systems  for  collecting  and  disseminating  applicable  diagnostic- 
related  information,  and  to  establish  requirements  and  incentives  for  acquiring  and  continuously 
improving  “designed-in”  MDC  capabilities.  In  conjunction  with  this  effort,  the  Task  Force  should 
assess  the  relative  benefits,  cost,  and  schedules  for  implementing  the  elements  of  the  improvement 
plan.  While  this  examination  is  on-going,  we  would  further  suggest  that  the  Services  not  start  any 
further  enhancements  to  MDC  systems  without  reviewing  the  synergistic  opportunities  of  more 
efficient  integrated  process  approaches  in  the  commercial  cases. 

5^  ACCELERATE  WEAPON  SYSTEM  SPECIFIC  APPLICATIONS 

DoD,  in  conjunction  with  the  above  plan,  should  ask  the  Services  to  accelerate  the 
demonstration  and  field  use  of  available  technology  solutions  for  diagnostic  improvement. 
Opportunities  to  apply  existing  diagnostic  improvement  technologies  should  focus  on  point-of- 
entry  maintenance  aiding,  transparent  data  collection,  centralized  diagnostic  performance 
analyses,  and  serial  tracking  and  maintenance  history  recording  of  reparable  items.  DoD  also 
should  provide  system  contraaors  with  the  latitude  to  propose  new  MDC  capabilities  or  add-ons 
to  existing  Service  systems  with  the  objective  of  increasing  diagnostic  capability.  Specific  near 
term  opportunities  identified  during  this  research  include  the  following: 

a.  Apply  commercial-off-the-shelf  (COTS)  technology  to  support  transparent  data  col¬ 
lection  capabilities  (i.e.,  bar-coding,  portable  memory/logic  microcircuit  devices). 

b.  Initiate  quick  reaction  research  efforts  to  demonstrate  the  feasibility  and  benefits  of 
integrating  reporting  mechanisms  across  product  and  ^pport  levels  (e.g.  investigate 
the  benefits  of  integrating  the  F- 18  parametric  ECAMS  data  with  NALCOMIS  and 
Aviation-3M  information). 

c.  Accelerate  implementation  of  the  F-15  pilot  debriefing  aid  (CFR)  system  and 
investigate  opportunities  to  apply  the  concept  to  other  weapon  system  operator 
debriefing  manuals. 

d.  Investigate  opportunities  to  automate  manuals,  fault  trees,  and  maintenance  processes 
on  new  and  existing  ATE  (i.e.,  down-sized  I-Ievel  ATE). 


5.3  ENHANCE  COMMON  MAINTENANCE  AND  SUPORT  SYSTEMS 


DoD  should  direct  the  Services  to  improve  the  responsiveness  and  performance  of  existing 
common  maintenance  and  support  systems.  Specific  opportunities  to  add  necessary  diagnostic 
in^rovement  functionality  identified  during  this  research  include  the  following: 

e.  Improve  up-time  and  responsiveness  of  common  MDC  systems  to  diagnostic  user 
needs  (i.e.,  reduce  the  excessive  down  time,  improve  the  user  friendly  characteristics, 
and  enhance  the  screen  responsiveness  of  CAMS). 

f.  Conduct  baseline  analyses  on  available  turbine  engine  data  to  expand  prognostic 
capabilities  (i.e.,  extend  the  full  prognostic  capabilities  of  CEMS  IV  and  ECAMS  to 
other  engine  types). 

g.  Link  MDC  information  sources  (i.e.,  adopt  design  changes  that  will  allow  CASS  to 
electronically  communicate  with  NALCOMIS). 

h.  Add  functionality  to  support  distribution,  codification,  storage,  and  analyses  of  data 
used  and  collected  on  front  end  information  sources  and  ATE  (i.e.,  increase  codified 
SAMS  maintenance  data  by  interfacing  directly  with  existing  and  proposed  ATE,  and 
automate  CASS  diagnostic  accuracy  analyses  and  feedback  between  I-  and  0-level 
maintenance). 


5.4  DEVELOP  DIAGNOSTIC  TECHNOLOGY 

DoD  should  establish  R&D  program  elements  to  support  technology  development  for 
diagnostic  improvement  processes  that  collect,  store,  analyze,  and  distribute  MDC  related 
information.  Specific  technology  needs  identified  during  this  research  include  the  following; 

a.  Integrated  maintenance  process  tools  to  support  and  manage  the  integration  of  MDC 
information. 

b.  Electronic  technical  information  presentation  capabilities  including  interface  and 
protocol  standards,  and  display  devices. 

c.  Configuration  tracking  and  reporting  tools  that  correlate  maintenance  procedures, 
testing  procedures,  parameter  ranges,  and  diagnostic  processes  aaoss  product  and 
maintenance  levels. 
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d.  Transparent  data  capture,  remote  data  recording  and  collection,  and  distribution 
technologies  from  electronic  BIT,  mechanical  fault  indicators,  and  on-  and  ofiF- 
equipment  ATE. 

e.  Diagnostic  analyses  tools  to  support  and  manage  both  local  and  central  diagnostic 
improvement  needs. 
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APPENDIX  A  -  DISCUSSION  OF  TERMS 

A.1  SYSTEM  DIAGNOSTICS 

System  diagnostics  as  used  within  the  context  of  this  paper  is  the  practice  of  analyzing  the 
cause  and  nature  of  a  fault  condition,  situation,  or  problem  and,  based  on  this  analysis, 
recommending  an  appropriate  course  of  action. 

a.  Conditions  relate  to  the  current  status  of  a  system  or  its  components.  Examples  of  con¬ 
ditions  include  mechanical,  chemical,  or  dielectric  changes  that  occur  due  to  exposure 
or  use  (e.g.  wear,  power  loss,  fatigue). 

b.  Situations  relate  to  the  operation  and  performance  of  a  system  under  varying  stress 
environments.  Examples  of  situations  are  intermittent  failures  or  electrical  power 
fluctuations  caused  by  external  stimuli. 

c.  Problems  relate  to  anything  from  a  “hard  failure,”  or  an  out-of-specification  parameter, 
to  a  costly  maintenance  action.  Examples  of  problems  include  electrical  shorts, 
in^roper  electrical  gain,  and  false  removal  of  a  subsystem  that  retests  OK  at  the  next 
maintenance  level. 

The  diagnostic  analyses  must  identify  the  problem  that  needs  to  be  resolved,  quantify  the 
irr^art  resulting  from  the  problem  (both  in  terms  of  operational  performance  as  well  as  the 
resulting  maintenance  and  support  burden),  and  identify  remedial  actions. 

K2  INTEGRATED  DIAGNOSTICS 

The  Automatic  Test  Committee  and  Logistics  Management  Committee  of  the  National 

Security  Industrial  Association  (NSIA)  defines  the  term  “integrated  diagnostics"  as 

a  structured  process  which  maximizes  the  effectiveness  of  diagnostics  by 
inte^ating  pertinent  elements  such  as  testability,  automatic  and  manual  testing, 
training,  maintenance  aiding,  and  technical  information  as  a  means  for  providing 
a  cost-effective  capability  to  detect  and  unambiguously  isolate  all  faults  known  or 
expected  to  occur  in  weapon  systems  and  equipment  in  order  to  satisfy  weapon 
system  mission  requirements  [Brown  1990,  p.80]. 

However,  participants  in  recent  joint  DoD  and  industry  workshops  on  “Opportunities  in 
Integrated  Diagnostics”  have  noted  that  the  term  is  used  in  several  different  contexts  and  often  with 
different  meanings:  to  describe  a  design  process  relating  to  the  integration  of  various  diagnostic 
elements,  describe  an  approach  for  acquiring  various  diagnostic  elements  as  a  package,  and 
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describe  isolated  deliverable  systems  that  provide  the  capability  of  integrating  the  benefits  of 
various  diagnostic  elements  [Brown  1990]. 

Consistent  with  the  NSIA  definition  as  well  as  the  observations  from  the  referenced 
workshops  is  the  notion  that  diagnostics  effectiveness  is  enhanced  by  the  syneigistic  application 
of  multiple  diagnostic  elements.  Therefore,  the  term  “integrated  diagnostics”  is  used  in  this  paper 
to  represent  the  process  of  improving  diagnostic  effectiveness  and  maintenance  efficiency  through 
the  integration  of  diagnostic  elements  that  promote  the  process  of  diagnostic  feedback,  analyses, 
procediu^  improvements,  design  evolution,  validation,  and  implementation. 


A3  MAINTENANCE  EFTICIENCY 

System  performance  effectiveness  is  a  combination  of  the  basic  system  capabilities,  the 
proficiency  of  operators,  the  availability  and  sustainability  of  the  system,  the  relative  cost  to 
operate,  the  maintainability,  the  resources  (trained  people,  equipment,  and  facilities)  needed  to 
maintain,  and  the  Support  cost  burden.  These  factors  are  neither  static  over  the  system  life  cycle, 
nor  are  they  independent  of  one  another.  A  fi'amework  for  integrating  diagnostic  elements  must 
have  the  ability  to  address  the  influence  that  each  of  these  factors  will  have  on  current  system 
performance  effectiveness. 

Modeling  techniques  necessary  to  fully  analyze  the  interrelationships  among  all  of  these 
factors  do  not  exist,  and  they  may  be  too  expensive  ever  to  develop.  However,  it  is  possible  to 
address  a  major  aspect  of  this  performance:  “maintenance  efficiency”  relative  to  maintenance  and 
support  costs.  Although  the  total  cost  for  optimum  maintenance  efficiency  is  not  known,  we  may 
assume  that  the  more  efficient  the  maintenance  and  support  process  (at  a  specified  level  of 
availability  and  for  a  given  tempo  of  operations),  the  lower  the  relative  maintenance  and  support 
costs.  Therefore,  for  a  given  level  of  availability  (generally  determined  by  specific  mission  needs), 
maintenance  efficiency  increases  as  the  cost  of  maintenance  and  support  decreases.  However,  the 
level  of  availability  and  the  maintenance/support  costs  are  not  independent  variables.  They  are 
influenced  by  many  of  the  same  conditions,  for  example,  the  proficiency  of  the  maintenance 
personnel. 


A.4  AVAILABLE  REMEDIAL  MAINTENANCE  ACTIONS 

Diagnostic  analyses  must  consider  available  actions  that  can  be  accomplished  at  different 
levels  within  the  operational  and  system  support  environments.  For  example,  actions  may  be 
performed  at  a  range  of  location;  the  system’s  operational  position,  any  of  a  series  of  intermediate 
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maintenance  facilities,  or  a  central  maintenance  depot.  Specific  remedial  actions  are  selected  from 
a  range  of  maintenance  actions  that  consider  the  available  resources  at  designated  locations,  the 
expertise  of  available  personnel,  the  urgency  to  remedy  the  problem,  and  the  relative  cost  to 
accomplish  any  action.  Generally,  recommended  remedial  actions  will  take  on  one  or  more  of  the 
following  forms. 

a.  Maintenance  Actions  to  Resolve  Hardware  Problems:  These  actions  may  include 
removing  and  replacing  components  (at  a  subsystem  or  component  level),  adjusting 
settings,  restoring  of  integrity  (tightening,  connecting,  soldering,  etc.),  and  cleaning. 

b.  Maintenance  Actions  to  Resolve  Software  Problems:  These  actions  may  include 
working  around  hardware  deficiencies  through  software  patches,  correction  of 
software  defects,  improving  code  efficiency,  and  retargeting  or  improving  applications. 

c.  Maintenance  Actions  to  Revise  Procedures:  These  actions  may  include  revising 
technical  manuals,  modifying  embedded  procedures  within  test  or  support  equipment, 
modifying  maintenance  concepts,  and  improving  training  practices. 

d.  Subsystem  or  Component  Redesign:  These  actions  may  occur  at  various  inaintenance, 
support  and  design  levels  depending  on  the  extent  of  the  design  modification.  These 
actions  are  generally  justified  based  on  the  results  of  operations  and  maintenance 
information  feedback. 


A,5  MAINTENANCE  DATA  COLLECTION 

MDC  is  the  process  of  collecting  maintenance  related  information,  and  sending  this 
information  to  a  repository  for  subsequent  analyses  or  management  planning.  With  computer 
technology  advances,  many  of  the  manual  MDC  processes  are  being  automated.  Figine  A-1 
highlights  common  architecture  surrounding  a  generalized  MDC  system.  Although  the  specific 
elements  as  well  as  the  modes  of  communication  will  vary  from  application  to  application,  this 
diagram  is  intended  to  establish  a  common  frame  of  reference  for  MDC  discussions. 

The  following,  identified  during  Integrated  Diagnostics  Workshop  [Brown  1990],  are  key 
MDC  application  attributes  that  influence  commercial  diagnostics  and  maintenance  efficiency 
inq)rovements. 

a.  Reduced  human  error  and  maintainer  workload  by  transparently  collecting  and  pro¬ 
cessing  information  (i.e.,  direct  input  from  a  data  transfer  imit  or  direct  input  from  auto¬ 
matic  test  equipment). 


37 


SUPPLY 


38 


Information  Links 

—  Electronic  Connection 
►  Manual  Interfaces 


b.  Collected  and  stored  system  performance  and  operational  status  for  future  analysis. 

c.  Applied  user-friendly  input  techniques  at  the  maintenance  interface  where  the  data 
input  could  not  be  automated. 

d.  Automated  local  analysis  of  current  and  historical  system  information  to  help  identify 
and  localize  maintenance  problems  (i.e.,  using  information  in  the  local  data 
repository). 

e.  Applied  implementation  approaches  that  focused  on  closed  loop  maintenance  data 
feedback  capability  through  a  centralized  organization. 

f.  Provided  rapid  central  analyses  capabilities  that  were  responsive  enough  to  influence 
on-going  production,  and  improve  current  and  future  maintenance  effectiveness. 

g.  Automated  configuration  management  of  products,  support  tools,  and  maintenance 
technical  manuals 


A.6  DUGNOSTICS  AND  MAINTENANCE  EFFICIENCY  IMPROVEMENTS 

Systems  design  approaches  for  diagnostic  and  maintenance  imptovement  are  needed 
which  feed  back  observed  problems,  develop  hypotheses  as  to  their  cause,  test  the  hypotheses, 
develop  and  test  proposed  solutions,  and  implement  corrective  actions. 

Improvements  to  diagnostics  and  mainteitance  efiiciency  are  directly  dependent  upon 
effective  maintenance  data  collection  and  feedback  to  a  responsible  agent  who  has  the  authority  to 
correct  a  deficiency.  The  following  illustrates  the  direct  dependence  that  diagnostics  and 
maintenance  efficiency  have  on  the  collection  and  analyses  of  maintenance  data. 

a.  If  maintenance  performance  data  are  not  collected,  trends  will  be  difficult  to  compute. 

b.  If  low  dit^ostic  performance  trends  are  not  noted,  problems  will  not  be  identified. 

c.  If  a  diagnostic  problem  has  been  identified,  but  not  to  the  responsible  authority,  it  will 
probably  not  be  fixed. 

d.  If  the  benefits  of  resolving  a  diagnostic  problem  can  not  be  quantified,  responsible 
authority  will  not  be  given  needed  resources. 

e.  If  the  solution  does  not  get  to  the  responsible  implementor,  the  benefits  will  not  be 
realized. 
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APPENDIX  B  -  SUMMARY  OF  SYSTEMS  AND  PROGRAMS 


EVALUATED 

The  following  sections  provide  an  overview  of  the  systems  and  programs  evaluated  during 
this  study.  Figure  B-1  summarizes  the  characteristic  repair  and  servicing  information  used  by  the 
Services.  At  present,  MDC  systems  used  by  the  three  Services  are  based  on  the  collection  and 
storage  of  this  repair  and  servicing  information.  This  information  is  then  sent  to  various  centralized 
data  repositories  for  subsequent  use  and  analyses. 

B.1  AIR  FORCE 

The  Air  Force  is  in  the  process  of  developing  and  implementing  several  integrated  MDC 
and  maintenance-related  information  systems.  Figure  B-2  illustrates  the  communication  interfaces 
between  these  major  systems. 


B.1.1  CAMS 

The  Core  Automated  Maintenance  System  (CAMS)  is  intended  to  automate  and 
standardize  the  Air  Force  base-level  maintenance  data  collection.  Each  CAMS  is  local  to  a 
maintenance  organization  (or  base)  and  sends  the  collected  information  to  a  central  repository. 
CAMS  is  accomplishing  this  by  automating  many  of  the  maintenance  information  reporting  forms 
and  by  permitting  the  maintenance  technician  to  enter  information  directly  into  the  system. 
Previously  these  forms  were  filled  out  by  the  maintenance  technician  and  then  later  keypunched 
into  various  maintenance  data  collection  systems  (the  specific  collection  systems  depended  on 
location  and  Air  Force  command). 

The  specific  type  and  range  of  collected  data  is  also  being  standardized  across  Air  Force 
maintenance  facilities,  and  many  of  the  maintenance  data  collection  systems  (Maintenance  Data 
Collection  (MDC)  system.  Maintenance  Management  Information  Collection  System  (MMICS), 
F-16  aircraft  Centralized  Data  System  (CDS),  and  B-IB  Configuration  Status  Accounting  System 
(CSAS),  etc.)  are  being  consolidated  into  CAMS. 

CAMS  is  increasing  the  relative  speed  of  data  collection  while  improving  the  accuracy 
and  accessibility  of  the  information  collected.  As  such,  it  is  improving  the  management  and 
utilization  of  existing  maintenance  resources  by  enhancing  and  standardizing  the  flow  and 
availability  of  automatic  data  processing  (ADP)  logistics  information. 


-  LEVEL  I -  LEVEL 


NEW  PARTS  FROM  |  {  INDUCTION  INTO 

SUPPLY  SYSTEM  I  - DEPOT 


The  first  Data  Project  Directive  for  CAMS  was  issued  in  May  1983.  Subsequent  to  that 
time  CAMS  has  become  the  standard  base-level  maintenance  system  for  aircraft,  engines,  trainers/ 
simulators,  support  and  test  equipment,  some  munitions,  and  communications-electronic 
systems.CAMS  can  hold  approximately  six  months  of  data-on-line  and  up  to  another  two  years  of 
data  on  tape.  It  is  hosted  on  the  standard  base  level  computer,  and  as  of  December  1989,  is  installed 
at  84  locations.  By  the  end  of  FY91  the  CAMS  Program  Office  at  the  Standard  Systems  Center, 
Gunter  AFB,  AL,  plans  to  have  a  total  of  113  systems  supporting  111  main/host  bases  and  153 
guard/reserves  sites. 

The  CAMS  program  has  evolved  through  a  series  of  planned  phases.  One  of  the  next 
plaimed  changes  to  CAMS  will  be  the  interface  (shown  on  a  dotted  line  on  figure  B-2)  to  the 
Standard  Base  Supply  System  (SBSS). 

Current  maintenance  data  collection  element  formats  are  based  on  Work  Unit  Codes 
(WUC).  The  data  elements  for  the  WUC  system  include  a  number  of  standardized  codes  such  as 
the  WUC  itself,  the  Action  Taken  Code,  How  Malfimction  Code,  When  Discovered  Code,  and 
Type  Maintenance  Code.  Many  <^f  the  maintenance  actions  reported  into  CAMS  require  the  serial 
number  and  the  part  number  of  the  item  undergoing  some  maintenance  action  and  the  next  higher 
level  of  assembly  thereby  designating  where  the  item  is  installed.  Other  information  that  is 
collected  and  stored  in  CAMS  includes  discrepancies  noted  by  operations.  Built-In-Tests  (BIT) 
fault  codes,  and  input  from  pilot  debriefing  sessions.  This  information  is  used  to  generate  a 
maintenance  work  orders.  From  this  information  evolves  requirements  for  other  information  in  the 
form  of  plans  and  schedules,  workload  tracking,  quality  control/quality  assurance,  spare  part 
requirements  and  tracking,  item/system  status  tracking,  and  management  tracking  and  progress 
reporting  data. 

The  data  collected  by  CAMS  is  used  for  maintenance  production,  control,  and  planning  at 
the  base  level.  In  addition  most  of  this  data  is  forwarded  to  Headquarters  Air  Force  Logistics 
Command  to  be  entered  into  the  logistics  infonnation  systems  used  to  support  the  logistics 
processes  of  maintenance  manpower  and  supply  requirements  definition,  acquisition,  distribution, 
and  maintenance. 

The  data  collected  by  CAMS  is  incident  to  the  maintenance  and  system  diagnostic 
processes,  and  not  intended  to  specifically  support  system  diagnostic  capabilities.  A  detailed 
description  of  the  data  collected  by  CAMS  and  the  specific  sources  of  this  data  may  be  found  in 
the  Maintenance  Data  Collection  User’s  Manual  (Air  Force  Manual  66-279,  Volumes  1-26).  In 
addition,  the  justification  and  rationale  behind  the  need  for  specific  data  may  be  foiu<d  in  the  Air 
Force  Technical  C)rder-0020  (series). 
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B.1.2  CEMS 


Comprehensive  Engine  Management  System  (CEMS)  is  an  on-line  database  developed  to 
provide  management  and  tracking  of  aircraft  turbine  engines  and  related  equipment.  Tt  >vas  dev  oped  by 
the  Air  Force  Irogistics  Command  to  satisfy  management  requirements  including  configuration 
management,  serialized  tracking,  Time  Compliance  Technical  Order  (TCTO)  tracking,  and  maintenance 
data  collection.  Several  conditions  lead  to  the  development  of  CEMS:  (1)  an  Air  Forcp  Irupector  General 
report  on  engine  management  in  December  1973  indicated  that  the  data  systems  were  not  responsive  to 
the  current  needs  and  that  these  needs  would  increase  with  the  new  F-15  and  A-10  aircraft  engines,  and 
(2)  in  the  mid  1970s  the  Secretary  of  Defense  directed  the  Air  Force  to  implement  a  concept  termed 
Reliability  Centered  Maintenance  (RCM)  that  requires  work  to  be  accomplished  based  on  need  rather 
than  predefined  times  or  cycles. 

CEMS  was  systematically  approached  in  increments.  Increments  I,  II,  and  III  are  completed. 
CEMS  IV  is  in  place;  however,  the  full  implementation  for  new  engines  and  bases  is  evolving.  Increments 
I  and  II  divided  the  implementation  of  critical-parts  tracking  for  different  engine  types.  Increment  III 
implemented  the  status  reporting,  inventory  control,  pipeline  analyses,  accounting,  distribution,  and 
requirements  computation  for  all  engines. 

CEMS  currently  tracks  approximately  70,000  units  (engines,  module,  APUs  etc.)  for  location, 
status,  configuration,  TCTTO  management,  and  installation.  In  addition,  there  are  a  number  of  engine 
components  and  accessories  (critical  safety  of  flight  and  time  change  items)  that  have  life  limits 
determined  by  engine  flight  hours  or  thermal  cycles.  The  CEMS  program  office,  located  at  Headquarters, 
Air  Force  Logistics  Command,  Wright  Patterson  AFB,  indicated  that  approximately  1 .4  million  serialized 
parts  are  tracked  and  this  niunber  equates  to  less  than  1  percent  of  all  the  serialized  items  on  engines.  AFM 
400-1  addresses  what  is  to  be  tracked  by  CEMS. 

The  data  capture  for  this  parts  life  tracking  (PLT)  covers  the  full  range  of  options:  manual 
recording  of  flight  hours  by  the  pilot,  manual  reporting  of  flight  hour  information  by  the  maintenance 
technician  based  on  meter  readings,  and  direa  electronic  down-load  of  flight  hour  readings  through  a 
portable  maintenance  aid.  The  maintenance  action  information  is  sent  to  CAMS  (or  MMICS)  and  the  rest 
of  the  data  is  sent  and  stored  at  the  CEMS  central  data  base  at  Tinker  AFB. 

CEMS  IV  is  intended  to  support  engine  diagnostics  and  trending  analyses.  It  accomplishes  this 
task  by  integrating  the  engine  performance  data  and  maintenance  data.  The  most  complete 
implementation  of  CEMS  IV  is  for  the  TF-34  engine  on  the  A- 1 0  aircraft.  The  Tlirbine  Engine  Monitoring 
System  (TEMS)  records  parametric  engine  performance  data  and  down-loads  to  CEMS.  This  information 
coupled  with  the  oil  analysis  program  (GAP),  component  life  expenditures,  and  other  maintenance  data 
already  collected  by  CEMS.  This  combined  information  is  used  to  develop  a  condition  profile  for  the 
engine.  This  profile  may  then  be  used  as  an  indicator  of  actions  required  on  that  engine. 


At  present,  only  the  T-34  engine  technical  data  has  been  analyzed  sufficiently  to  yield  the 
full  range  of  prognostic  and  diagnostic  opportunities.  Efforts  to  analyze  and  organize  the  technical 
data  for  other  engines  that  posses  this  parametric  monitoring  capability  are  continuing.  At  present, 
the  engine  maintenance  technicians  are  not  able  to  use  this  diagnostics  capability  to  its  fullest 
extent  since  they  are  not  always  certain  what  the  trend  data  they  are  collecting  actually  indicates. 


B.1.3  REMIS 

Reliability  and  Maintainability  Information  System  (REMIS)  will  be  an  on-line 
centralized  database  for  status  and  inventory  reporting,  configuration  accounting,  and  reliability 
and  maintainability  performance  data.  The  program  was  started  in  1983  and  was  established  as  a 
Headquarters  Air  Force  directed  program.  The  purpose  of  REMIS  is  to  put  in  place  a  stanoard  Air 
Force  system  to  collect  essential  data  to  support  reliability  and  maintainability  at  all  levels  of  the 
Air  Force.  Its  goal  is  to  inaease  readiness  and  sustainability  of  USAF  weapon  systems  by 
improving  availability,  accuracy  and  flow  of  essential  equipment  maintenance  information. 
REMIS  will  provide  real-time  access  to  Air  Force  equipment  and  maintenance  information  for 
analyses  of  trends,  excessive  maintenance  costs,  and  quality  problems. 

When  completed,  REMIS  will  provide  an  on-line  distributed  data  base  for  weapon  system 
product,  maintenance,  configuration,  utilization,  and  status  data.  The  availability  of  the  data  will 
permit  automated  analyses  and  immediate  access  to  “user  defined”  reports  for  studies  and 
analyses.  REMIS  will  consist  of  four  subsystems:  (1)  Equipment  Inventory,  Multiple  Status  & 
Reporting  Subsystem  (EIMSURS),  (2)  Product  Performance  Subsystem  (PPS),  (3)  Generic 
Configuration  Status  Accounting  Subsystem  (GCSAS),  and  (4)  MICAP  (Mission  Capable 
Awaiting  Parts)  and  Awaiting  Parts  Subsystem  (MAPS). 

EIMSURS  will  provide  inventory  control,  status  reporting,  and  equipment  utilization  by 
serial  number.  Most  of  the  items  tracked  by  this  subsystem  arc  major  inventory  end-items.  The 
specific  items  to  be  tracked  are  identified  in  APR  65-110,  and  REMIS  will  provide  real-time  feed 
back  relative  to  “assigned  to”  mformation.The  status  reporting  will  identify  system  problems 
(multiple  problems  if  they  exist,  not  just  the  top  problem).  The  utilization  by  serial  number  will, 
among  other  things,  help  support  the  budget  development.  The  initial  operating  capability  (IOC) 
of  EIMSURS  occurred  in  December  1989. 

PPS  will  collect  and  store  information  from  various  Air  Force  maintenance  data  collection 
systems.  The  objective  is  to  have  on-line  one  year’s  record  of  on-&  off-  equipment  maintenance 
data  as  well  as  summarized  data  for  the  past  five  years.  Off-line  it  will  have  the  capability  of 
retrieving  all  transactions  for  the  life  of  the  specific  inventory  item,  plus  five  years.  Specific 
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information  the  PPP  will  store  include  (1)  Full  maintenance  data  history,  (2)  Maintenance 
deficiency  reports  and  service  reports,  and  (3)  Actual  performance  information  as  well  as  Logistics 
Support  Analysis  Record  (LSAR)  and  Time-Compliance  Technical  Order  (TCTO)  predictions. 
This  information  may  be  used  for  problem  item  identification,  unit  to  unit  performance 
comparison,  and  warranty  performance  monitoring.  The  IOC  of  PPS  is  projected  for  September 
1990. 

GSCAS  will  provide  configuration  status  accoimting  from  the  component,  to  the  Shop 
Replaceable  Unit  (SRU),  to  the  Line  Replaceable  Unit  (LRU),  to  the  system  level.  It  will  also 
include  the  configuration  program  identification  number  (CPIN)  for  the  required  software.  GCS  AS 
will  track  both  hardware  and  software  configuration  items  and  identify  what  is  authorized  and  what 
is  actually  installed.  GSCAS  also  will  provide  for  the  management  and  tracking  of  TCTO’s.  The 
specific  contractual  delivery  date  for  GCSAS  has  not  been  established  yet;  however,  the  program 
office  at  Wright  Patterson  AFB,  OH  anticipates  IOC  of  this  subsystem  during  the  summer  of  1992. 

MAPS  will  tie  together  the  status  of  the  systems  being  tracked,  the  configuration  of  needed 
parts,  and  the  process  of  tracking  needed  parts.  It  will  identify  and  isolate  MICAP  items,  provide 
analyses  of  MICAP  items,  and  provide  status/tracking  information  for  needed  parts.  Funding  for 
the  development  of  this  subsystem  was  in  the  FY91  President’s  budget.  If  fimded,  it  will  take 
approximately  26  months  to  develop. 


B.1.4  DMMIS 

Depot  Maintenance  Management  Infonnation  System  (DMMIS)  is  a  resource 
management  and  planning  system  being  developed  for  depots.  DMMIS  is  intended  to  replace 
twenty-nine  existing  management  systems,  and  to  collert  and  control  depot  maintenance  planning, 
management,  and  production  control  information.  The  host  computer  is  Thndem  and  the  program 
is  applying  commercial  off-the-shelf  software  packages  through  a  series  of  steps. 

The  first  step  incluuv.s  the  development  of  a  data  base  to  support  the  bill-of-materials, 
routing,  and  workorders.  This  portion  is  currently  in  operational  and  acceptance  testing.  The 
second  step  includes  the  development  of  an  interface  to  supply  and  will  provide  subsystems  to 
support  material  control.  This  portion  of  the  DMMIS  system  is  currently  being  coded  and  initial 
delivery  for  testing  is  anticipated  late  spring  1990.  Step  3  will  incorporate  the  planning  portions  of 
Manufacturing  Resources  Planning  (MRP  II).  Anticipated  completion  is  late  summer  1990.  Step 
4  will  include  the  shop-floor  and  cost  management  capabilities.  This  portion  is  anticipated  around 
the  end  of  the  year. 


DMMIS  will  first  be  installed  and  tested  at  Ogden  Air  Logistics  Center,  Hill  AFB  UT  to 
support  depot  maintenance  of  the  Technical  Repair  Center  (TRC)  for  landing  gear.  The  DMMIS 
system  will  then  be  further  refined  to  support  other  TRCs  and  installed  at  other  Air  Logistics 
Centers  within  the  Air  Force.  The  next  projected  refinement  will  support  the  engine  depot  TRC. 

In  order  to  fiilly  understand  the  function  and  purpose  of  DMMIS,  it  was  necessary  to 
understand  the  generic  process  for  item  repair  within  a  TRC  depot  environment  The  landing  gear 
depot  repair  processes  were  used  as  examples. 

A  repairable  end-item  is  received  at  the  depot  It  may  be  ( 1 )  a  full  subsystem  that  is  tracked 
as  a  field  replaceable  end-item,  (2)  a  sub-component  that  is  tracked  as  replaceable  end-item  in  a 
field  environment,  or  (3)  individual  parts  that  are  removed  and  replaced  in  the  field  and  are  tracked 
as  end-items.  If  the  end-item  is  an  assembly,  it  will  go  through  the  necessary  disassembly 
operations  in  preparation  for  the  detailed  inspection,  refurbishment,  or  repair  as  required.  Various 
inspections  are  required  at  each  stage  both  before  and  after  process  work.  The  results  from  this 
flow  are  (1)  condemned  parts  and  requests  to  supply  for  new  replacement  resources  and  (2)  good 
parts  that  may  be  put  back  into  a  holding  inventory. 

If  the  end-item  is  not  an  assembly,  it  will  go  through  the  appropriate  inspection  and  repair 
as  required.  The  results  from  this  flow  are  similar. 

Most  serialized  end-items  lose  their  identity  and  traceability  after  the  incoming  inspection. 
There  are  exceptions  for  some  components  (such  as  some  safety  of  flight  items,  selected  engine 
components,  etc.),  and  in  these  rare  cases,  the  items  are  tracked  by  their  serial  number  even  within 
the  depot.  These  are  exceptions,  generally  only  quantities  of  specific  end-items  that  are  introduced 
in  the  depot  repair  process  are  tracked.  As  replacement  needs  for  condemnations  or  consumable 
spare  parts  are  identified,  new  parts  are  requisitioned  from  the  supply  system. 

If  an  end-item  is  serialized  controlled  in  the  field,  the  new  serial  number  is  not  assigned 
imtil  the  end-item  status  is  achieved  (note  the  multiple  states  that  an  end-item  may  have  as 
discussed  above:  end-item  assembly,  sub-component,  etc.).  There  is  an  extremely  low  probability 
that  the  original  parts  of  a  specific  assembly  will  be  aggregated  and  recombined  in  the  refurbished 
end-item. 

DMMIS  is  intend  to  help  plan  and  support  the  coordination  (negotiations  for  work  to  be 
accomplished)  between  the  item  managers  and  the  depot  The  depot  facilities  need  to  account  for 
capacity  (facilities  and  people),  spare  part  needs,  through-put,  material  control,  etc.  DMMIS  will 
provide  the  data  base  to  track  quantities  of  the  respective  end-items  against  historical  work  load, 
maintenance  action  averages,  and  available  capacity.  It  will  also  permit  the  depot  to  maintain 
maintenance  inventory  records,  compute  material  requirements,  and  schedule  shop-floor 
activities.  The  maintenance  action  data  collected  will  be  stored  in  REMIS. 
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Computerized  Fault  Reporting  CFR  is  a  prototype  concept  that  was  initially  developed 
and  managed  by  personnel  at  the  Air  Force  Logistics  Manager.cnt  Center  (LMC)  at  Gunter  AFB, 
AL  The  prototype  CFR  is  a  computer  program  shell  that  will  run  on  the  Air  Force  standard 
personal  computer.  The  CFR  was  not  illustrated  on  Figure  B-2. 

The  initial  thrust  for  this  concept  began  back  in  1983  when  Tactical  Air  Command  (TAC) 
noticed  that  the  pilots  and  maintenance  persormel  were  not  using  the  trouble-shooting  technical 
order  for  the  F-15  aircraft.  These  trouble  shooting  technical  orders,  referred  to  as  debriefing  books, 
were  intended  to  take  advantage  of  the  BIT,  cockpit  fault  and  caution  lights,  and  pilot  noted 
problems.  Using  this  information  these  debriefing  books  would  systematically  carry  the  pilot  and 
maintenance  technician  through  a  comprehensive  fault  tree  to  the  specific  source  of  a  problem. 

TAC  discovered  that  these  debriefing  books  fell  into  disfavor  for  several  reasons: 

( 1 )  The  books  were  too  complicated  and  took  too  long  to  follow  through  to  the  end. 

(2)  The  fault  tree  was  based  on  early  information  and  many  of  the  fault  codes  were 
incorrea  (i.e.,  immature). 

(3)  There  was  inadequate  feedback  as  to  fault  tree  problems  and  when  discovered 
there  was  no  easy  method  to  correct  (on  average  it  takes  up  to  270  days  to  update 
the  TO.) 

As  a  part  of  this  study,  McDonnell  Douglas  computerized  (automated)  the  fault  tree 
structure.  It  is  interesting  to  note  that  this  process  alone,  identified  and  forced  the  contractor  to  fix 
approximately  350  logic  errors  and  diagnostic  voids  in  the  fault  tree.  The  CFR  was  hosted  on  the 
Air  Force  standard  computer  (Z-248),  and  effectively  becomes  a  front  end  processor  to  CAMS. 
Based  on  the  available  BIT,  fault  logging  information,  and  the  pilots  input,  the  CFR  progresses 
through  an  automated  fault  tree.  Once  completed,  that  data  is  entered  directly  into  CAMS  and  the 
necessary  work  orders  are  generated  to  fix  the  identified  problem. 

The  prototype  Ch'k  demonstrated  more  accurate  fault  assessment,  decreased  debriefing 
time,  and  faster  turnaround  of  aircraft  forms.  As  a  result  of  the  study  and  demonstration,  the  F-15 
SPO  has  decided  to  complete  development  of  the  CFR  and  field  this  diagnostic  support  tool  with 
the  F-15E  aircraft. 

The  LMC  personnel  expressed  one  concern  regarding  the  CFR.  Based  on  early  analysis/ 
projections  they  expect  that  the  identified  disCTepancies  noted  during  the  debriefing  to  increase  by 
a  factor  of  2  to  4  initially.  They  projected  that  this  will  be  due  to  the  improved  diagnostics  and  the 
presence  of  previously  hard  to  locate  problems.  However,  once  fully  mature,  and  the  relatively 


high  level  of  previously  unidentified  discrepancies  have  been  corrected,  LMC  personnel  believe 
that  overall  maintenance  performance  and  efficiency  will  improve  significantly. 

Finally,  they  noted  that  there  were  still  opportunities  to  further  enhance  this  concept  by 
integrating  approaches  that  would  permit  the  direct  down  loading  of  BIT  and  fault  logging  data 
into  a  data  transfer  unit.  This  would  eliminate  the  potential  of  error  (remove  the  “air-gap”)  while 
decreasing  the  maintenance  technician’s  workload. 


B.1.6  RAM  &  TMRS 

The  Air  Force  has  the  Technical  Order  Reliability  Asset  Monitoring  (RAM)  program  for 
tactical  missile  systems.  The  program  is  managed  out  of  Wamer-Robins  Air  Logistics  Center 
(WRALC).  The  guiding  documentation  (and  operating  instructions)  are  found  in  Technical  Order 
21M-1-101  (3  January  90,  Change  2).  This  Technical  Order  implements  the  Tactical  Missile 
Reporting  System  (TMRS)  that  is  in  place  at  the  various  field  locations.  The  TMRS  consists 
primarily  of  a  computer  program  that  operates  on  a  personal  computer  in  the  missile  maintenance 
shop.  The  required  reliability  information  (including  the  identified  fault  codes  from  the  respective 
test  sets)  against  each  serial  numbered  item  are  entered  in  a  data  base.  The  data  base  is  sent  on  a 
monthly  basis  via  floppy  disk  to  WRALC. 

The  current  TMRS  system  originated  as  a  result  of  the  combined  efforts  of  HQ  TAC  and 
the  LMC  at  Gunter  AFB.  The  individual  that  has  been  responsible  for  the  management  and  control 
of  the  program  up  until  recently  has  been  stationed  at  HQ  TAC.  However,  this  individual  will  be 
transferring  (along  with  the  responsibility  to  continue  the  TMRS  management)  to  WRALC.  This 
is  intended  as  an  interim  approach  until  a  new  system  called  the  Combat  Ammunition  Supply 
System  (CASS)  is  developed  and  fielded.  Concepts  being  looked  at  for  CASS  include,  bar  coding 
of  part  number  (P/N)and  serial  number  (S/N)  data,  and  on-line  reporting. 

Due  to  the  basic  design  of  most  of  the  missile  systems,  very  little  detailed  repair  of  end- 
items  (major  missile  components)  is  performed  in  the  field  environments  (the  classical  I-  &  O- 
levels  of  maintenance).  The  current  operational  cOTcept  is  moving  towards  a  “Wooden-Round” 
approach  where  the  field  maintenance  is  generally  limited  to  inspections,  high  level  functional 
testing  (either  BIT  or  with  a  test  set),  aircraft  loading,  and  remove  and  replacement  of  major  end- 
items  (if  applicable).  Beyond  these  responsibilities,  the  missile  maintenance  technician  handles  the 
missile  as  though  it  were  a  wooden-round.  As  a  result  of  this  maintenance  concept,  coupled  with 
the  fact  that  many  of  the  components  are  serialized  ontrolled  and  tracked  over  their  life  cycle  (due 
to  reasons  associated  with  explosives  or  parts  classification),  the  information  recorded  in  the  field 
under  the  TMRS  is  directly  traceable  to  the  depot  repair  actions.  Tlie  ciirrent  RAM  program 
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currently  tracks  the  observed  field  problem  (BIT  code,  test  set  fault  code,  inspection  discrepancy, 
etc.)  for  each  returned  item  and  compares  this  data  with  the  actual  depot  repair  actions. 


B.2  NAVY 

B,2.1  NAMP(A-3MandNALCOMIS) 

The  overall  Navy  (Air)  maintenance  data  collection  effort  is  covered  by  the  Naval 
Aviation  Maintenance  Program  (NAMP),  OPNAVINST  4790.2E,  dated  1  January  1989.  The 
process  is  paper  intensive  and  maintenance  action  reporting  is  done  on  5-part  (carbon)  forms  called 
Visual  Information  Display  System/Maintenance  Action  Form  (VIDS/MAF).  These  forms  track 
the  specific  mainteiuince  action  by  a  job  control  number,  (JCN),  the  system  identification  (tail 
number,  serial  number,  etc.),  type  of  problem,  malfunction  code,  action  taken,  etc.  If  parts  are 
removed  and  sent  to  another  area  or  organization  for  inspection,  testing  or  repair  additionalVIDS/ 
MAP  are  generated.  The  fiow  and  process  involving  these  forms  is  illustrated  in  Figures  B-3  and 
B-4.  VIDS/MAF  forms  are  also  sent  to  a  Data  Service  Center  (or  equivalent)  for  key  punch  entry 
of  the  data.  At  shore  facilities  this  is  often  accomplished  by  a  support  contractor.  On  an  aircraft 
carrier,  this  function  is  generally  performed  by  personnel  on  board  the  ship.  The  data  (referred  to 
as  aviation  3M  data)  captured  in  this  process  is  asnt  via  tape  to  the  Naval  Maintenance  Support 
Office  (NMSO)  in  Mechanicsburg  PA.  NMSO  sends  reports  and  summary  data  to  program 
managers  and  to  Air  Systems  Command  as  well  as  the  Naval  Aviation  Logistics  Data  Analysis 
(NALDA)  System.  Summary  reports  of  compiled  VIDS/MAF  data  are  also  sent  from  the 
individual  Data  Services  Facilities  (or  equivalent  facilities)  to  the  respective  I,  O,  or  D  level 
maintenance  organization  that  submitted  the  forms. 

At  NALDA  the  data  is  further  processed  and  selected  portions  are  put  on  line  for  fleet¬ 
wide  use.  Typical  users  include  Fleet  Comnumders,  depot.  Type  Commanders  (TYCOMs)  and  the 
Systems  Command. 

The  Naval  Aviation  Logistics  Command  Management  Information  System  (NALCOMIS) 
supports  the  NAMP  and  automates  the  maintenance  data  collection  (3M  data)  discussed  above.  It 
is  being  implemented  as  a  computerized  (aviation  maintenance  management)  information  system 
that  will  facilitate  direct  data  entry,  auto  information  validation,  and  timely  update  and  retrieval  of 
essential  data.  The  initial  phase  of  the  NALCOMIS  program  is  being  implemented  at  I-level  and 
is  focusing  on  the  supply  system  interface. 


DATA  SERVICES  FACILITY 


Figure  B-4.  Processing  of  Navy  VIDS/MAF  Forms. 
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BJL2  ECAMS 


The  Enhanced  Comprehensive  Asset  Monitoring  System  (ECAMS)  is  an  engine  and 
aircraft  asset  tracking  and  management  system.  ECAMS  includes  a  capability  to  down-load 
aircraft  and  eiigine  in-flight  parametric  data  through  a  data  storage  unit  (DSU)  on  the  aircraft.  The 
information  is  then  used  by  the  technician  for  parts  life  tracking  engine,  prognostics,  etc. 

At  present  the  Navy  has  several  aircraft  that  can  down-load  engine  and  aircraft  in-flight 
data.  Table  B-1  lists  these  aircraft  and  asset  reporting/monitoring  systems.  The  unshaded  areas 
represent  deployed  operational  capability. 


Table  B-1.  Enhanced  Comprehensive  Asset  Monitoring  System  (ECAMS)  and  Other  Engine 

Monitoring  System 


A/C 

Down  Load 
Medium 

System 

A-7 

Tape  Cartridge 

Engine  Monitoring  System  Ground  Station  (EMSGS) 

F/A  - 18 

TTM/ DSU* 

ECAMS  (with  Martin  Marietta  software) 

• 

F- 14 

DSU 

ECAMS  (withGE  software) 

E-2C 

Tape  Cartridge 

EMSGS 

AV-8B 

DSU 

Map,  Operator,  Maintenance  Station  (MOMS) 

% 

V-22 

DSU 

VMARS 

A- 12 

DSU 

(TBD) 

*  Early  models  of  the  F/A-1 8  have  a  tape  transport  magazine  (TTM)  later  models  have  the 

(DSU). 


The  EMSGS  and  its  onboard  monitoring  capability  in  the  A-7  is  reported  to  have  saved  at 
least  22  aircraft.  Although,  there  is  no  way  of  conclusively  verifying  this  benefit,  it  was  projected  ^ 

based  on  the  most  probable  result  had  the  pilot  or  maintenance  technician  not  received  the  critical 
information  captured  by  the  EMSGS  system.  This  system  is  used  primarily  at  the  0-level  and 
captures  engine  event  data.  Based  on  the  data,  the  technician  will  direct  specific  actions  (i.e., 

Require  an  inspection  based  on  an  over  temperature  or  over  speed  condition).  The  information,  as  • 

presently  applied  to  the  engine  in  the  A-7  aircraft,  is  not  used  for  parts  life  tracking  and  this  data 
is  not  stored  or  retained. 
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The  application  and  use  of  ECAMS  on  the  F-18  is  split  between  the  I-  and  O-  levels.  The 
information  collected  by  the  DSU  (and  TTM  on  the  earlier  aircraft  models)  is  extensive  and  may 
be  loosely  grouped  into  4  categories:  (1)  mission  data,  (2)  engine  data,  (3)  strain  gauge  data,  and 
(4)  avionics  data.  All  of  the  shore  based  F-18  ECAMS  are  networked  together,  and  as  such,  the  F- 
18  aircraft  has  the  only  centralized  engine  parts  life  tracking  (PLT)  capability  in  the  Navy.  The 
DSU  is  generally  removed  from  the  aircraft  when  it  is  80%  full  and  is  not  used  for  each  post-flight 
analysis.  In  an  operational  scenario,  the  DSU  is  removed  more  frequently;  however,  it  goes  to  the 
operations  organizations  first  to  extract  the  mission  data.  Therefore,  the  information  is  seldom 
available  for  immediate  maintenance  diagnostics  applications.  A  summary  of  the  type  of  F-18 
ECAMS  information  that  fall  in  categories  2,  3  and  4  is  provided  in  table  B-2. 

The  strain  gauge  data  along  with  data  concerning  the  number  of  landings  and  amount  of 
flight  time  are  collected  via  the  ECAMS  and  put  on  a  Navy  form  called  the  “  Yellowsheet”  or  Flight 
Readiness  Evaluation  Data  Sheets  (FREDS).  This  information  is  sent  to  the  Naval  Air 
Development  Center  to  monitor  and  compute  the  fatigue  life  extension  (FLE)  on  the  individual 
aircraft.  At  present  some  individuals  question  the  integrity  of  the  strain  gauge  data.  One  suggested 
source  of  error  originates  from  the  faa  that  the  strain  gauges  are  not  considered  prime  mission 
equipment  (PME),  and  as  such,  the  aircraft  is  authorized  to  be  flown  without  strain  gauges 
connected. 
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Table  B-2.  Summary  of  F/A-18  ECAMS  Data  Types 

1.  Strain  Data:  High  and  low  stress  values  for  the  strain  sensors  on  board  aircraft. 

2.  Life  Use  Index  (LUI)  Data:  The  LUI  values  are  accrued  during  flight  for  the 
engines  installed  in  the  aircraft. 

3.  Pre-Event  Data:  Data  shows  engine  parameters  1,  5  &  10  seconds  preceding  a 
detected  engine  event 

4.  Avionics  Data:  Shows  the  BIT  for  Avionics  equipment  on  board  the  aircraft: 

a.  DMS:  Digital  Map  Set 

b.  RMDI  &  LMDI:  Right  &  Left  Multipurpose  Display  Indicators 

c.  NFLR:  Navigation  FLR 

d.  SDC:  Signal  Data  Computer 

e.  DSU:  Data  Storage  Unit 

f.  HF  Comm:  High  Frequency  Communication 

g.  ASPJ:  Airborne  Self-protection  Jammer 

h.  CWS:  Countermeasures  Warning  System 

i.  ADC:  Air  Data  Computer 

j.  MMD:  Master  Monitor  Display 

k.  MFD:  Multifunction  Display 

l.  CSC:  Communication  System  Control 

m.  INS:  Inertial  Navigation  Set 

n.  SMS:  Stores  Management  System 
0.  FLR:  Forward  Looking  Radar 

p.  RDR:  Radar  Set 

q.  LST:  Laser  Spot  Tracker 

r.  FCA  &  FCB:  Flight  Control  Computers  A  &  B 

s.  HARM:  High  Speed  Anti-Radiation  Missile 

t.  MSDRS:  Maintenance  Signal  Data  Recording  Set 

u.  D/L:  Data  Link 

V.  Comm  1  &  Comm  2:  Communications  Radios  1  &  2 

23. Tactical  Air-To-Air  &  Air-To-Ground  Data:  Aircraft  and  weapon  parameters  for 
launches 

24. FIR  Data:  In-flight  aircraft  parameters  recorded  1/second. 

25. Post-Event  Engine  Data;  Shows  engine  parameters  1  &  10  seconds  following  an 
event 

26.  Hard  Landing  Data:  Vertical  and  normal  accelerations,  and  weight  for  current 
and  hardest  landings. 

27. SDC  Fuel  Data:  Fuel  system  data  if  fuel  system  event  is  detected. 

28. AOA/Q  Matrix:  Time  spent  at  various  angles  of  attack  and  dynamic  pressures. 

29.  Engine  Exceedingly  Reports;  Pre-  and  post-  engine  event  data  for 
troubleshooting. 


The  engine  data  is  used  primarily  for  PLT.  The  data  is  automatically  collected  and  sent  to 
a  central  data  base.  The  software  as  currently  in^lemented  for  the  F- 18  aircraft  engines  (GE:  F- 
404)  does  not  support  diagnostics  nor  trending  analyses.  In  addition  to  the  PLT,  the  information  is 
currently  used  by  the  managers  to  select  engines  with  lower  time  for  extended  cruses,  to  identify 
appropriate  modules/asset  for  swapping  and  change-out  based  on  operating  time,  and  to  flag 
critical  event  data  that  might  drive  a  maintenance  action. 

All  of  the  avionics  BIT  flags  and  flight  parameters  that  are  being  recorded  by  the  DSU  are 
archived  with  the  ECAMS.  However,  the  use  of  this  data  for  analyses  has  been  minimal  to  date. 
The  primary  exceptions  are  for  (1)  post  mishap  analyses,  (2)  an  engine  mission  profile  study  by 
GE,  and  (3)  a  recent  study  into  strain  gauge  creep. 

The  ECAMS  capability  for  the  F- 14  engines  (A+  and  D  upgrades  with  the  new  GE:  F-110 
engines)  has  only  been  available  for  about  6  months.  It  currently  supports  both  1-  and  0-  level; 
however,  the  I-  level  is  not  on  line.  The  engine  PLT  information  is  forwarded  to  GE  for 
management  and  tracking.  The  GE  software  for  the  ECAMS  supports  Trend  and  Event 
Diagnostics  System  (TEDS)  on  the  F-110  engine.  The  F-14  depot  has  requested  a  data  link  to  the 
central  data  base. 

The  EMSGS  capability  on  the  E-2C  is  very  similar  to  the  A-7.  It  has  been  in  place  only 
for  about  6  months  (since  the  black  box  was  added  to  the  T-56/427  engine). 

The  systems  for  the  AV-8B  and  the  V-22  will  be  storing  the  information  in  a  database  that 
will  provide  user  sort  capabilities.  The  MOMS  system  (on  the  AV-8B)  is  designed  around  a  man- 
portable  (80386  microprocessor  based)  field  system  that  will  support  three  separate  functions:  map 
station,  operator  station,  and  maintenance  station.  The  software  loaded  on  the  hard  drive  and  the 
selection  of  peripheral  devices  wiU  determine  the  specific  station  function. 

The  A- 12  is  projected  to  have  two  DSU  like  devices  to  support  the  ECAMS  functions.  The 
reported  intent  is  to  separate  the  mission  data  from  the  maintenance  data. 

At  present  the  Navy  is  attempting  to  pursue  a  program  to  standardize  the  ECAMS  software 
for  the  F-18  and  F-14  aircraft.  They  are  focusing  on  the  GE  software  with  the  TEDS  capability.  In 
addition,  they  are  also  investigating  options  to  field  a  standard  personal  computer  compatible 
software  version  to  supplement  the  approximate  110  ECAMS  computers  (PDPll)  now  deployed. 


B.2.3  AWAR 

The  Navy  has  in  place  an  Airborne  Weapon  Analyses  and  Reporting  (AWAR)  system  for 
tracking  and  monitoring  the  reliability,  maintainability,  availability  and  quality  of  the  deployed 
missile  systems.  The  program  is  documented  and  controlled  by  the  Naval  Airborne  Weapons 
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Maintenance  Program  (NAWMP),  OPNAVINST  8600-2A,  dated  1  Mar  89.  It  is  currently  used  to 
monitor  approximately  twelve  Navy  missile  systems. 

At  present,  the  maintenance  technicians  record  the  information  on  paper  forms.  The  Fleet 
Tactical  Analyses  Center  is  investigating  options  to  transition  to  a  personal  computer  (PC)  based 
data  collection  system  that  will  permit  using  bar  code  readers.  This  is  being  tested  at  several 
weapon  stations. 

The  type  of  information  collected  includes  the  following;  date  and  time  of  maintenance, 
the  operator,  item  nomenclature,  P/N,  S/N,  source  code  (where  it  came  from  and  why  sent  -  “fleet 
unknown”,  captive  carry,  etc.),  Lot  No.,  test  times,  test  results  (including  BIT,  parametric 
performance  information,  etc.),  present  condition  codes,  disposition,  technical  directive  applied, 
failure  category  codes,  and  test  equipment  used.  This  data  is  then  tracked  for  the  life  of  the  missile, 
along  with  the  configuration  and  repair  and  rework  data  from  the  depot.  A  significant  number  of 
components  in  most  of  the  missiles  are  tracked  by  S/N  or  P/N  and  Lot  No.  Therefore,  the  AWARs 
data  base  has  the  capability  of  tracking  all  actions  and  major  components  of  individual  missiles 
over  the  life  cycle.  This  data  when  coupled  with  actual  live  firing  testing  (with  telemetric  data) 
provides  the  Navy  with  a  comprehensive  data  base  for  detailed  analyses. 

BJ.4  CASS 

The  Consolidated  Automated  Support  System  (CASS)  will  update  and  replace  the  Navy’s 
current  suite  of  automatic  test  equipment.  CASS  will  be  located  at  the  I-level  maintenance  facility 
and  will  be  used  to  detect  and  fault  isolate  date  subsystem  and  circuit  card  problems  and  to  verily 
that  corrective  maintenance  actions  have  resolved  problems.  The  program  is  currently  in  Tech 
Eval/Op  Eval  at  Patuxent  River,  with  the  Low  Rate  Initial  Production  (LRIP)  decision 
programmed  for  the  August  1990  time  frame.  Option  1  of  the  contract  will  acquire  99  units  with 
an  approximate  16  month  delivery  time  for  the  first  units.  The  full  rate  production  is  projected  for 
the  1992  time  period  and  there  are  3  years  of  contract  options. 

The  CASS  system  will  automate  the  VIDS/MAF  form  and  the  identified  fault  codes  will 
be  entered  into  the  forms  "Discrepancy”  box  automatically.  The  CASS  system  will  incorporate  a 
bar-code  reader  and  the  reader  will  be  used  to  identify  the  part  number  and  serial  number  of  the 
following:  the  unit  tmder  test  (UUT  -  box,  circuit  card,  etc.),  the  interface  device  (ID)  needed  to 
cormect  the  UUT  to  the  CASS,  any  required  cables,  and  the  test  program  sets  (TPS).  The 
embedded  software  will  check  the  compatibility  of  each  selected  item  for  the  specific  test  and  will 
record  this  information  for  future  analyses  if  required.  The  CASS  system  is  being  sized  to  store 
approximately  6  months  of  collected  maintenance  data. 
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CASS  will  incorporate  paperless  technical  manuals  that  are  resident  in  the  system.  In 
addition,  the  CASS  system  is  designed  to  fault  detect  and  fault  isolate  itself.  All  of  the  CASS 
stations  will  be  networked  together,  thereby  creating  one  single  data  base  of  the  stored 
maintenance  information,  configuration,  documentation,  and  technical  manual  information. 
Preliminary  indications  are  that,  due  to  the  embedded  training  and  technical  information 
capabilities,  training  time  for  both  the  technician  and  the  CASS  maintainer  will  be  1/2  to  1/3  that 
of  the  existing  systems. 

At  present,  the  CASS  design  does  not  permit  direct  communication  with  the  NALCOMIS 
system.  The  VIDS/MAF  forms  must  be  generated  (automatically  produce  paper  forms)  by  the 
CASS,  and  then  the  information  must  be  keypunched  into  either  the  3M  or  NALCOMIS  data  base. 
The  program  office  noted  that  the  direct  electronic  link  was  eliminated  due  to  budget  and  program 
constraints;  however,  key  interfaces  that  will  permit  easy  modification  have  been  retained  in  the 
system  specification.  Finally,  since  at  a  specific  location  the  CASS  stations  are  all  networked 
together  the  program  office  does  not  believe  that  potential  modifications  to  incorporate  this 
automated  communication  between  CASS  and  NALCOMIS  will  be  significant 


BJ.5  lETM 

The  Navy  is  investigating  opportunities  to  use  Interactive  Electronic  Technical  Manuals 
(lETM).  David  Taylor  Research  Center  (DTRC)  is  the  Navy  representative  to  a  tri-Service 
committee  that  is  chartered  to  develop  the  requirements,  standards,  and  specifications  for  an  lETM 
type  device  that  delivers  interactive  electronic  presentation  of  technical  manual  information.  These 
specifications  are  expected  to  include  hardware,  software  and  data  standards  that  define  the 
functional  and  content  requirements  needed  to  meet  operational  and  maintenance  requirements. 
Earlier  tests  were  conducted  under  the  Navy’s  NTIPS  (Navy  Technical  Information  Presentation 
System)  program  that  preceded  this  current  effort.  In  order  to  demonstrate  the  feasibility  of  the 
concept,  the  NTIPS  program  conducted  a  series  of  controlled  field  tests  on  sea-  and  air-systems 
using  both  experienced  as  well  as  inexperienced  maintenance  technicians.  The  following 
summarizes  the  results  of  these  tests. 

( 1 )  Tests  comparing  the  use  of  paper  based  technical  manuals  and  electronic  technical 
information  presentations  to  maintenance  technicians  demonstrated  increased 
maintenance  accuracy  that  was  directly  attributable  to  the  use  of  electronic  pre¬ 
sentations.  The  maintenance  tasks  involved  the  rudder-trim  system  of  the  F-14A 
aircraft  100%  of  the  test  subjects  (12  out  of  12)  successfully  located  the  faults 
with  the  electronic  presentation  while  only  43%  (5  out  of  12)  were  successful  with 
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paper  technical  information.  In  addition,  there  were  35%  fewer  errors  by  the  inex¬ 
perienced  technicians  who  used  the  electronically  delivered  technical  informa¬ 
tion,  “The  electronic  Delivery  of  Automated  Technical  Information  For  Logistics 
Support  of  Navy  Weapon  Systems;  Potential,  Systems:  Potential,  System 
Description,  and  Status”,  Feb  1989,  [DTRC-89/007]. 

(2)  Tests  were  conducted  for  the  AN/SPA-25D  shipboard  radar  repeater  in  a  mainte¬ 
nance  shop.  Again  100%  of  the  technicians  correctly  fault  isolated  the  problem 
using  the  electronic  presentation  (11  experienced  and  13  inexperienced  techrti- 
cians).  However,  only  58%  of  the  technicians  using  the  paper  technical  instruc¬ 
tions  (7  inexperienced  and  7  experienced)  were  able  to  isolate  the  fault  without 
help  from  the  test  monitors.  The  test  also  revealed  that  the  trouble  shooting  time 
was  twenty-four  times  faster  with  the  electronic  display  than  with  paper.  [DTRC- 
89/007] 


BJ  ARMY 

B3.1  ULLS  AND  SAMS 

The  Unit  Level  Logistics  System  (ULLS)  is  a  personal  computer  based  system  that  is 
located  at  the  unit  maintenance  (0-)  level  and  is  used  to  both  manage  and  request  support  for 
actions  that  are  beyond  the  capability  of  the  unit  level  maintenance.  ULLS  successfully  completed 
its  milestone  3  (production  decision)  this  year  and  is  replacing  the  old  “paper-forms”  with  floppy 
disks.  When  using  the  ULLS,  paper  is  used  only  for  backup,  and  ULLS  communicates  via  ue  disks 
both  to  the  next  higher  level  of  maintenance  support  (the  Direct  Support  or  General  Support  Units) 
and  to  the  Standard  Army  Retail  Supply  System  (SARSS).  Information  pertaining  to  the  specific 
requests  (status,  inventory,  priority,  etc.)  is  sent  both  directions  (to  and  from  ULLS)  via  these  disks. 

The  Standard  Army  Maintenance  System  (SAMS)  is  an  automated  maintenance 
management  system  that  is  hosted  on  the  Tactical  Army  Combat  Computer  System  (TACCS). 
SAMS  milestone  3  was  approved  in  FY  86  and  the  active  duty  Army  implementation  is  essentially 
complete.  It  replaces  several  old  maintenance  management  systems  including  the  Maintenance 
Reporting  and  Management  (MRM)  System,  and  the  Maintenance  Activity  Management  System 
(MAMS).  There  are  two  different  software  implementations  of  SAMS.  SAMS-1  is  used  at  the 
direct  support  maintenance  company  found  in  the  separate  brigade,  division,  corps,  and  echelons 
above  corps;  and  the  general  support  maintenance  company  at  echelons  above  corps.  SAMS  is 
used  to  improve  the  management  of  maintenance  actions,  work  loads,  and  resources.  SAMS 
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currently  receives  inputs  via  the  disks  originating  from  ULLS  and  has  the  capability  to 
communicate  with  other  systems  using  the  standard  phone  lines  and  DDN  communication  links. 

SAMS-2  is  used  at  the  forward  support  battalion,  main  support  battalion,  and  material 
management  center  of  the  division  and  separate  brigade.  SAMS-2  is  used  at  these  various 
maintenance  management  levels  to  enhance  program  guidance.  It  accomplishes  this  by  controlling 
and  coordinating  maintenance  work  load  functions,  monitoring  mission  capable  rates,  and 
coordinating  repair  parts  utilization  to  maximize  equipment  availability. 

Data  in  SAMS  can  be  accessed  instantly  to  fulfill  management  needs  in  controlling, 
coordinating,  reporting,  analyzing  and  reviewing  maintenance  actions.  Information  from  SAMS- 
2  is  also  sent  to  the  Army  Material  Readiness  Support  Activity  (MRSA)  The  maintenance 
infomoation  flow  and  the  maintenance  action  flow  are  illustrated  on  Figures  B-5  and  B-6. 

Until  the  introduction  of  SAMS,  the  Army  relied  primarily  on  a  series  of  individual  study 
efforts  to  collect  operational  system  performance  and  availability  information.  These  efforts  are 
grouped  under  a  program  titled  “San^le  Data  Collection”  (SDC).  Each  of  these  efforts  is  relatively 
short  term,  focuses  on  a  single  system,  and  results  in  a  summary  report  The  SDC  information 
collected  is  of  little  value  for  providing  continuous  maintenance  feedback  information  to  support 
diagnostic  improvement 

By  contrast,  SAMS  provides  a  continuous  feedback  capability  and  provides  insight  into 
the  system  and  subsystem  reliability  and  availability  status.  In  addition,  this  new  information  is 
collected  as  a  beneficial  by-produa  of  the  SAMS  maintenance  management  support  function. 
However,  the  SAMS  information  will  not  presently  support  effective  diagnostics  analyses.  The 
level  of  maintenance  information  detail  is  significantly  less  than  the  information  details  collected 
by  either  the  Navy  or  Air  Force  MDC  systems. 

Although,  useful  feedback  potential  of  SAMS  diagnostic  related  information  is 
significantly  less  than  the  MDC  systems  in  the  Navy  and  Air  Force,  SAMS  demonstrates  the 
potential  of  applying  current  technology  to  a  functional  need  and  developing  a  new  process  that  is 
not  based  on  a  paper  MDC  concept.  The  demonstrated  by-products  of  the  SAMS  approach  are  (1) 
a  system  that  supports  the  users  needs,  (2)  collects  maintenance  related  data  (that  was  only 
available  with  the  SDC  program)  without  increasing  workload,  and  (3)  provides  the  medium  for 
collecting  more  detailed  diagnostics  related  information  in  the  future. 
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